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This  research  and  development  was  conducted  within  Marine  Corps  Air-Ground 
Technology  Program  Element  No.  62131M  under  the  Training  Technology  Project  No. 
CF31T33,  Task  Area  01  and  under  the  sponsorship  of  Headquarters,  United  States  Marine 
Corps,  Code  TAP. 

The  objective  of  this  effort  was  to  provide  recommendations  for  the  development  of 
military  training  decision  aids.  This  report  examines  and  critiques  23  existng  training 
decision  aids  and  identifies  major  issues  involved  in  determining  training  situations/levels 
and  selecting  training  media. 

The  report  is  intended  for  the  use  of  instructional  developers  and  managers. 


B.  E.  BACON 
Captain,  U.S.  Navy 
Commanding  Officer 
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Technical  Director 
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SUMMARY 


Objectives 

The  objectives  of  this  effort  were  to:  (1)  examine  major  issues  involved  in  the 
establishment  of  training  situations/levels  and  the  selection  of  training  media,  (2)  review 
and  compare  the  characteristics  of  decision  aids  developed  to  assist  training  managers, 
developers,  and  other  users  to  establish  training  situations/levels  and  select  training 
media,  and  (3)  provide  recommendations  for  the  development  of  training  decision  aids 
oriented  to  military  training  requirements. 

Background 

The  military's  System  Approach  to  Training  includes  requirements  for  specification  of 
the  situations/levels  at  which  training  is  to  take  place  and  the  appropriate  training  media. 
At  present,  such  specifications  must  often  be  made  without  adequate  guidance  and 
documentation.  As  a  result,  important  factors  can  be  overlooked  in  the  decision  process. 
Additionally,  it  is  often  difficult  to  track  and  record  the  decision  process  for  such 
specifications  in  order  to  justify  decisions  or  evaluate  decision  criteria. 

Problems 


A  wide  variety  of  training  decision  aids  are  now  available  to  assist  military  decision 
makers  in  determining  training  situations/levels  and  selecting  training  media.  Analyses  of 
these  aids  are  needed  to  determine  the  appropriateness  of  their  formats  and  procedures. 

Approach 

First,  publications  examining  the  effectiveness  of  training  decision  aids  were 
reviewed  and  existing  training  decision  aids  were  analyzed.  Second,  major  issues  involved 
in  determining  training  situations/levels  and  selecting  training  media  were  identified. 
Existing  training  decision  aids  were  characterized  and  compared  with  respect  to  these 
issues.  Third,  recommendations  were  developed  for  tailoring  training  decision  aids  to 
meet  the  needs  of  military  users. 

Findings  and  Conclusions 

1.  Most  existing  training  decision  aids  have  not  been  found  to  be  effective  in  aiding 
personnel  to  select  training  situations/levels  or  media. 

2.  Desirable  attributes  for  military  training  decision  aids  can  be  identified  and 
justified  from  reviews  of  current  research  and  examinations  of  existing  aids. 

3.  Existing  decision  aids  for  determining  relataive  costs  of  training  paths  and 
refresher  training  requirements  appear  to  be  effective  adjuncts  to  aid  in  the  process  of 
establishing  training  situations/levels. 

4.  None  of  the  reviewed  aids  satisfy  all  of  the  required  attributes  identified  in  this 
report.  However,  many  existing  training  decision  aids  contain  desirable  criteria  that 
could  be  incorporated  into  more  effective  aids  that  would  be  suitable  for  military 
requirements. 


vn 


J 


rA 


f 


5.  Methods  for  designing  more  effective  military  training  decision  aids  that 
incorporate  desirable  selection  criteria  are  needed. 

Recommendations 


These  recommendations  provide  a  basis  for  guiding  the  design  and  development  of 
military  training  decision  aids  that  can  be  used  to  improve  the  quality  of  training 
decisions  and  provide  cost/effective  training  programs. 

1.  Develop  training  decision  aids  for  specific  military  applications. 

2.  Combine  computerized  flowcharts  and  matrix  formats  in  structuring  decision 

aids. 

3.  Provide  detailed  guidance  for  decision  makers. 

4.  Develop  training  decision  aids  for  specific  types  of  tasks. 

5.  Select  training  situations/levels  based  on  task  prerequisites,  training  priorities, 
available  training  time,  task  appropriateness,  and  training  costs. 

6.  Use  existing  situation/level  decision  aids  for  determining  relative  costs  and 
refresher  training  intervals. 

7.  Select  training  presentation  methods  based  on  training  location,  group  character¬ 
istics,  training  events,  stimulus  characteristics,  affective  (motivational)  requirements, 
level  of  difficulty,  programming  requirements,  group  practice,  simulation  requirements, 
and  requirements  for  automation  of  testing  and  data  management. 

8.  Select  training  media  based  on  training  presentation  methods;  requirements  for 
modification,  mobility,  and  development  time;  costs;  availability;  and  instructor 
preference. 

9.  Consider  a  combination  of  training  presentation  methods  and  media  for  teaching 
each  task. 
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INTRODUCTION 


Background 

In  accordance  with  the  Systems  Approach  to  Training  (SAT),  military  training 
organizations  are  adopting  systematic  approaches  to  training  development.  These 
approaches  include  specifying  the  training  situations/levels  at  which  training  is  to  take 
place  and  selecting  appropriate  training  media.  Situation/level  specifications  indicate 
where  the  training  should  be  provided  (in  formal  schools  or  at  unit  commands),  when  it 
should  be  provided  (during  preliminary,  advanced,  or  refresher  training),  and  who  should 
receive  the  training  (what  occupations,  grade  levels,  and,  where  appropriate,  levels  of 
maintenance  responsibility).  Training  media  specifications  indicate  the  media  delivery 
systems  that  should  be  used  for  specific  types  of  training  situations/levels,  methods,  and 
content. 

At  present,  military  decision  makers  must  often  determine  training  situations/levels 
and  select  training  media  without  benefit  of  systematic  consideration  and  documentation 
of  the  many  factors  that  influence  training  decisions.  As  a  result,  important  factors  may 
be  excluded  from  the  decision  process.  Additionally,  without  a  systematic  procedure,  it  is 
often  difficult  to  track  and  record  the  decision  process  to  justify  the  decisions  or  to 
evaluate  the  appropriateness  of  the  decision  criteria. 

Objectives 

The  objectives  of  this  effort  were  to: 

1.  Examine  major  issues  involved  in  the  establishment  of  training  situations/levels 
and  the  selection  of  training  media. 

2.  Review  and  compare  the  characteristics  of  decision  aids  developed  to  assist 
training  managers,  developers,  and  other  users  to  establish  training  situations/levels  and 
select  training  media. 

3.  Provide  recommendations  for  the  development  of  effective  training  decision  aids 
oriented  to  military  requirements. 

Problems 


A  wide  variety  of  training  decision  aids  are  now  available  to  assist  military  decision 
makers  in  determining  training  situations/levels  and  selecting  training  media.  However, 
no  analyses  of  these  aids  have  been  conducted  to  determine  the  appropriateness  of  their 
formats  and  procedures. 


APPROACH 


Method 


This  effort  started  with  a  review  of  (1)  publications  on  the  effectiveness  of  training 
decision  aids  and  (2)  existing  decision  aids  for  determining  training  situations/levels  and 
selecting  training  media.  Next,  the  major  issues  involved  in  determining  training 
situations/levels  and  selecting  training  media  were  identified.  Existing  training  decision 


aids  were  characterized  and  compared  with  respect  to  these  issues.  Finally,  recommenda¬ 
tions  were  developed  for  tailoring  training  decision  aids  to  meet  the  needs  of  military 
users. 

Review  Orientation 


The  present  review  of  decision  aids  for  establishing  training  situations/levels  and 
selecting  training  media  differs  from  previous  reviews  in  a  number  of  ways.  First, 
previous  reviews  have  only  dealt  with  decision  aids  for  media  selection  and  have  paid 
relatively  little  attention  to  aids  for  determining  training  situations/levels. 

Second,  past  reviews  have  typically  been  organized  either  in  terms  of  the  media 
characteristics  the  aids  deal  with  (Reiser  &  Gagne,  1982)  or  in  terms  of  the  media 
selection  aids  themselves  (Heidt,  1978;  Levie,  1977).  Both  forms  of  organization  have 
some  merit.  A  consideration  of  media  characteristics  provides  a  basis  for  discussing  what 
features  are  beneficial  to  a  decision  aid.  A  description  of  individual  decision  aids 
facilitates  the  determination  of  how  the  various  characteristics  are  incorporated  into 
each  aid  and  allows  potential  users  to  decide  which  type  of  decision  aid  might  be  most 
beneficial  for  their  particular  requirements.  Therefore,  this  report  provides  both  forms  of 
organization.  Characteristics  of  training  decision  aids  are  discussed  in  the  body  of  this 
report  and  each  decision  aid  is  described  in  detail  in  Appendix  A. 

Third,  in  attempting  to  characterize  media  selection  issues,  past  reviews  have 
typically  confounded  presentation  methods  with  media.  The  distinction  between  presenta¬ 
tion  and  media  is  considered  to  be  a  critical  factor  in  media  selection  and  is  maintained  in 
the  present  review. 

Fourth,  in  reviewing  training  decision  aid  characteristics,  past  reviews  were  often 
noncommittal  on  the  value  of  the  various  approaches  that  were  taken  in  developing 
decision  aids.  This  report  attempts  to  assess  the  value  of  these  different  approaches  in 
order  to  provide  a  basis  for  recommendations  for  the  development  of  training  decision 
aids. 


Fifth,  the  review  emphasizes  training  decision  aids  and  related  issues  that  are 
relevant  to  military  users. 

Report  Organization 

The  next  section  presents  research  findings  on  the  effectiveness  of  training  decision 
aids,  discusses  the  characteristics  of  existing  aids,  critiques  the  appropriateness  of 
different  formats  and  criteria  used  in  these  aids,  and  provides  recommendations  for 
developing  effective  training  decision  aids  for  military  users. 

Summary  descriptions  of  existing  training  decision  aids  referenced  in  this  report  are 
presented  in  Appendix  A.  A  cross-reference  to  training  decision  aids  recommended  for 
use  or  as  examples  is  provided  in  Appendix  B. 


A  glossary  of  special  terms  used  in  this  report  is  provided  on  page  29. 
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FINDINGS  AND  DISCUSSION 


Effectiveness  of  Training  Decision  Aids 


Research  Findings 


From  an  initial  exploration  of  the  research  findings,  one  finds  little  support  for  the 
use  of  training  decision  aids.  With  respect  to  decision  aids  for  establishing  training 
situations/levels,  no  studies  have  been  conducted  to  determine  their  effectiveness.  With 
respect  to  decision  aids  for  media  selection,  the  results  of  experimental  evaluations  have 
been  mixed. 

The  most  extensive  evaluation  of  media  selection  aids  was  conducted  by  Braby  (1973). 
In  Braby's  study,  the  subjects  selected  media  for  a  specific  course  content  based  on  one  of 
nine  media  selection  models  or  based  on  their  own  independent  opinions.  Each  of  the 
models  (including  independent  opinions)  was  then  rated  by  two  evaluators  based  on  their 
subjective  judgments  of  the  effectiveness  of  the  resulting  media  decisions.  While  some 
models  received  higher  ratings  than  others,  none  received  higher  ratings  than  were  given 
to  independent  opinions.  None  of  the  evaluated  media  selection  aids  produced  more 
effective  media  choices  than  were  made  without  aids. 

Braby's  negative  findings  can  be  questioned  in  terms  of  the  expertise  of  the  subjects 
who  used  the  decision  aids.  Five  of  Braby's  six  subjects  were  experienced  in  training 
system  design.  A  knowledgeable  user  who  is  already  familiar  with  a  wide  range  of 
instructional  methods  and  media  and  their  proper  applications  might  not  benefit  from 
guidance  as  much  as  would  a  less  sophisticated  decision  maker. 

This  position  was  supported  in  a  later  study  by  Higgins  and  Reiser  (1985).  The  authors 
randomly  assigned  one  of  four  different  media  design  problems  to  each  of  22  in¬ 
experienced  graduate  students.  First,  the  students  selected  the  best  medium  or  media  for 
the  assigned  design  problem  based  on  their  intuition.  Next,  the  students  reviewed  an 
article  describing  characteristics  of  media  selection  models  (Reiser  <5c  Gagne,  1982)  and  a 
portion  of  a  text  on  the  use  of  a  media  selection  flowchart  designed  by  Reiser  and  Gagne 
(1983).  Again,  each  student  was  randomly  assigned  one  of  the  four  media  design  problems 
and  required  to  select  the  best  medium  or  media.  Two  instructional  designers  with 
"considerable  design  experience"  had  previously  identified  the  "correct"  media  solutions 
for  the  four  design  problems.  Results  indicated  that,  17  percent  of  the  students  who  used 
their  intuition  chose  only  the  correct  medium,  while  39  percent  of  the  students  who  used 
the  flowchart  chose  only  the  correct  medium. 

Braby's  negative  findings  can  also  be  questioned  in  terms  of  effectiveness  of  the 
media  selection  aids  he  evaluated.  The  ratings  indicated  that  some  of  the  aids  resulted  in 
less  effective  media  choices  than  those  made  by  intuition.  Such  findings  cause  one  to 
question  the  formats  and  criteria  employed  by  the  evaluated  aids.  Braby  does  not 
comment  on  the  appropriateness  of  the  formats  or  criteria  used  by  the  media  selection 
aids  he  evaluated.  Furthermore,  his  descriptions  of  those  aids  are  not  detailed  enough  for 
the  reader  to  determine  whether  or  not  the  nature  and  content  of  the  aids  are 
appropriate. 

Questions  remain,  then,  as  to  what  characteristics  of  media  selection  aids  might  be 
expected  to  lead  to  effective  media  choices  and  whether  the  design  of  existing  aids  takes 
advantage  of  these  characteristics.  These  issues  will  be  discussed  in  detail  in  the  Analysis 
of  Training  Decision  Aids  section. 


Military  Requirements 

Even  if  effective  training  decision  aids  for  establishing  training  situations/levels  and 
selecting  media  can  be  developed,  they  might  not  appear  useful  to  military  users  for  a 
number  of  reasons.  First,  aids  designed  for  civilian  users  may  not  be  appropriate  for 
military  personnel.  Some  criteria  that  might  have  to  be  considered  for  diverse  civilian 
populations  (e.g.,  blindness)  would  not  apply  to  military  trainees. 

Second,  to  be  useful  to  the  military,  the  training  decision  aids  must  be  appropriate  in 
terms  of  the  user's  background  and  capabilities.  Some  aids  may  not  be  useful  to  military 
users  because  of  unfamiliar  terminology.  Also,  aids  that  are  not  prescriptive  but  merely 
list  factors  to  be  considered  may  not  provide  sufficient  guidance  for  military  users  who 
are  relatively  unfamiliar  with  training  issues. 

Third,  some  training  decision  aids  may  be  inappropriate  for  certain  military  users  due 
to  the  complexity  of  their  requirements.  Military  users  often  have  to  make  decisions 
relevant  to  a  wide  variety  of  course  work  in  a  short  period  of  time.  No  matter  how 
effective  an  aid  might  be,  users  probably  will  not  adopt  it  if  its  procedures  are  excessively 
complex  and  not  "user  friendly,"  or  if  it  requires  information  not  readily  available. 

Finally,  a  training  decision  aid  that  would  be  useful  to  one  military  user  may  not  be 
useful  to  another.  Military  users  make  training  decisions  under  a  variety  of  conditions.  In 
media  selection,  for  example,  a  course  instructor  may  have  to  select  media  from  a  limited 
choice  of  options  based  on  what  is  currently  available.  A  training  designer  who  must 
decide  whether  to  use  a  simulator  for  training  personnel  to  operate  a  new  weapons  system 
has  quite  a  different  problem. 

Nontraining  Benefits 

The  use  of  training  decision  aids  may  provide  benefits  in  addition  to  more  effective 
training  decisions.  For  example,  the  use  of  systematic  decision  procedures,  established  in 
training  decision  aids,  might  facilitate  the  tracking  and  justification  of  training  decisions. 
Such  procedures  would  be  particularly  useful  when  training  decisions  have  to  be  defended 
or  justified  to  higher  authorities  such  as  congressional  review  boards. 

Summary 

There  is  a  strong  possibility  that  properly  designed  training  decision  aids  could 
improve  the  effectiveness  of  military  decision  makers  in  determining  training  situa¬ 
tions/levels  and  selecting  training  media  and  aid  them  in  tracking  and  justifying  training 
decisions.  To  be  optimally  effective,  however,  such  aids  must  be  appropriately  designed 
and  oriented  to  the  needs  of  specific  users.  The  elements  of  appropriate  design  will  be 
discussed  in  detail  in  the  following  section. 

Analysis  of  Training  Decision  Aids 


This  section  reviews  the  characteristics  and  structure  of  decision  aids  that  have  been 
developed  to  assist  personnel  in  establishing  training  situations/levels  and/or  selecting 
media.  The  requirements  of  training  decision  aids  are  discussed  and  the  relative 
effectiveness  of  approaches  utilized  in  developing  such  aids  are  considered. 

Of  the  23  training  decision  aids  reviewed,  20  are  designed  to  aid  in  selecting  training 
media;  2,  in  establishing  training  situations/levels;  and  1,  in  both  processes.  All  of  these 
training  decision  aids  were  developed  or  revised  between  1965  and  1987.  Nine  of  them 
were  developed  by  or  for  military  users. 
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The  specific  decision  aids  reviewed  in  this  report  are  listed  and  described  in  Appendix 
A.  These  aids  are  also  identified  and  listed  in  the  reference  section. 

Characteristics 

Previous  reviews  of  training  decision  aid  characteristics  have  generally  centered  on 
media  selection  and  excluded  training  situations/levels.  Each  review  considered  a 
different  list  of  characteristics.  Braby  (1973)  considered  Learning  Task  Categories,  Task 
Difficulty,  Task  Frequercy,  Stage  of  Learning,  Learning  Guidelines/Strategy,  Stimulus 
Characteristics,  Student  Response  Modes,  Student  Feedback  Characteristics,  Individuali¬ 
zation  Capability,  Economic  Analysis,  Sensitivity  to  Changing  Requirements,  Time  to 
Achieve  Criterion,  Threshold  for  Media  Requirement,  and  Multiple  Practice  Factors. 
Reiser  and  Gagne  (1982)  considered  the  following  media  selection  model  characteristics. 
Display  Form  and  Mode  of  Decision  Making;  Categories  of  Media;  Physical  Attributes 
(visuals,  printed  words,  sound,  motion,  color,  and  real  and  simulated  objects);  Learner, 
Setting,  and  Task  Characteristics;  and  Practical  Factors  (such  as  production  costs  and 
hardware  availability). 

The  present  review  considers  a  somewhat  more  extensive  list  of  characteristics  than 
was  previously  examined,  based  on  the  large  variety  of  training  decision  aids  that  are  now 
available.  Also,  on  occasion,  consideration  is  given  to  important  characteristics  and 
interactions  among  characteristics  that  are  not  directly  discussed  in  any  of  the  reviewed 
decision  aids. 

Training  decision  aids  can  be  characterized  on  several  dimensions  including  the  type 
of  application  intended  for  the  aid,  the  format  of  the  decision  aid,  the  level  of  user 
guidance  provided,  the  training  unit  at  which  the  aid  is  directed  (task,  objective,  etc.),  and 
the  criteria  used  in  making  decisions.  The  following  discussion  considers  each  of  these 
dimensions  separately. 

Applications 

There  are  a  variety  of  applications  for  military  training  decision  aids.  Specific  types 
of  applications  for  military  aids  reviewed  in  this  report  are  listed  below. 

Assisting  Military  Personnel  to  Establish  Training  Situations/Levels.  An  aid  for 
establishing  initial  training  situations/levels  was  developed  by  Pieper,  Guard,  Michael,  and 
Kordek  (1978);  an  aid  for  comparing  the  cost  effectiveness  of  different  training  sites  was 
developed  by  Collins,  Hernandez,  Ruck,  Vaughn,  Mitchell,  and  Rueter  (1987);  and  an  aid 
for  determining  refresher  training  intervals  was  developed  by  the  Army  (User's  Manual  for 
Predicting  Task  Retention,  1985). 

Assisting  Military  Personnel  to  Select  Media.  Aids  for  assisting  instructors  to  select 


media  were  developed  by  the  Marine  Corps  (The  Marine  Corps  Systems  Approach  to 


Training  User's  Guide  (Draft),  1987)  and  the  (Methods/Media  Selection  Guidelines,  1977 
An  aid  for  assisting  course  designers  or  planners  to  select  media  was  developed  by  Braby 
(1973).  An  aid  for  assisting  the  replacement  of  conventional  instruction  with  new 
technologies  was  developed  by  the  Air  Force  (Nonpersonnel  Studies  and  Analysis  Services 
for  Assessment  of  New  Training  Technologies.  1985). 

Few  of  the  aids  reviewed  in  this  report  are  adequately  designed  for  military  users. 
Some  address  criteria  that  are  inappropriate  for  the  military.  Many  use  terminology  that 
military  users  would  probably  not  understand.  Most  are  either  too  complex  for  personnel 
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who  are  not  experts  in  training  design  or  lack  sufficient  guidance.  Even  aids  specificaliy 
designed  for  military  applications  are  seldom  designed  for  specific  military  users.  This 
can  be  a  problem  in  several  ways.  Users  may  have  different  levels  of  choices  available  to 
them.  One  user  may  have  to  choose  from  a  small  range  of  available  media  while  another 
may  be  able  to  select  from  the  broadest  range  of  possibilities.  Also,  different  information 
concerning  costs,  facilities,  and  student  populations  may  be  available  to  different  military 
users.  To  be  effective,  military  training  decision  aids  should  be  tailored  to  the  resources 
that  are  available  to  the  intended  user. 

Formats 

Reiser  and  Gagne  (1982),  in  their  review  of  media  selection  models,  identified  three 
basic  types  of  display  formats  used  for  guiding  the  user:  flowcharts,  matrices,  and  work 
sheets.  According  to  the  authors,  a  flowchart  generally  provides  a  progressive  narrowing 
of  media  choices  while  a  matrix  or  worksheet  typically  defers  choices  until  all  selection 
criteria  are  considered.  In  the  present  survey,  it  was  found  that  some  decision  aids 
combine  display  formats  or  use  them  with  written  outlines  or  instructions. 

Although  Reiser  and  Gagne  did  not  discuss  the  relative  merits  of  these  different 
formats,  each  has  its  own  advantage.  The  flowchart  format  is  useful  for  reducing  the 
number  of  choices  to  consider.  The  matrix  format  is  useful  for  comparing  a  number  of 
possibilities  with  respect  to  a  single  criterion  (e.g.,  costs).  The  worksheet  is  typically 
used  in  a  manner  similar  to  the  matrix  but  with  the  user  providing  the  variables  and/or 
rating  the  variables  on  selected  criteria.  The  flowchart  and  matrix  formats  appear  to 
provide  users  with  a  maximum  amount  of  guidance.  It  might  be  an  advantage  to  combine 
flowchart  and  matrix  formats.  First,  flowcharts  could  be  used  to  reduce  the  number  of 
choices  to  a  manageable  number;  then,  matrix  formats  could  be  used  to  compare 
remaining  choices. 

None  of  the  reviewed  training  decision  aids  combine  flowcharts  and  matrices. 
However,  examples  of  flowchart  formats  are  provided  by  Anderson  (1983),  Bretz  (1971), 
Levie  (1975),  Reiser  and  Gagne  (1982),  and  Romiszowski  (1974).  Examples  of  matrix 
formats  are  provided  by  Biaby  et  al.  (1975),  Holden  (1974),  the  Navy  (Method/Media 
Selection  Guidelines,  1977),  and  Walker  (1965). 

Several  of  the  reviewed  decision  aids  are  designed  for  use  with  computers. 
Computers  appear  to  be  particularly  helpful  for  speeding  calculations  and  manipulating 
complex  data.  Menu-driven  computers  could  probably  be  used  in  conjunction  with 
flowcharts,  matrixes,  or  worksheets  to  increase  their  efficiency. 

A  computerized  automated  aid  for  determining  the  cost  effectiveness  of  alternative 
training  sites  was  developed  by  Collins  et  al.  (1987).  A  computerized  aid  that  is  based  on 
the  automation  of  procedures  for  determining  refresher  training  was  developed  by  the 
Army  (The  User's  Manual  for  Predicting  Task  Retention,  1985). 

One  of  the  reviewed  media  selection  aids  was  designed  for  use  with  a  computer 
(Kribs,  Simpson,  &  Mark,  1983).  However,  as  indicated  later  in  this  report,  this  media 
selection  aid  provides  minimal  user  direction. 

Guidance 

Each  of  the  types  of  formats  used  for  training  decision  aids  can  provide  different 
levels  of  guidance.  A  flowchart  can  simply  identify  a  decision  point  or  it  may  prescribe  a 
choice  based  on  task  characteristics.  A  matrix  format  may  rate  media  on  specified 
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criteria  that  the  user  is  to  consider  or  it  may  require  the  user  to  provide  the  ratings. 
Some  worksheets  provide  the  user  with  lists  of  media  to  rate  on  different  criteria.  Others 
require  the  user  to  determine  the  media  and/or  criteria  that  will  be  rated. 

Training  decision  aids  that  require  users  to  rate  potential  situations/levels  or  media 
also  provide  different  levels  of  guidance  for  the  rating  procedure.  Some  decision  aids 
simply  require  the  user  to  place  a  check  mark  beside  each  criterion  that  a  particular 
medium  satisfies  and  to  total  the  number  of  checks.  Other  aids  require  the  user  to  rate 
how  well  each  medium  satisfies  each  criterion.  The  user  must  average  these  ratings  to 
obtain  an  overall  score  for  each  medium.  Such  ratings  provide  a  more  sensitive 
comparison  than  check  marks.  Still  other  aids  require  the  user  to  assign  a  weighting  to 
each  criterion  according  to  its  importance.  In  this  case,  the  overall  score  is  an  average  of 
the  ratings  multiplied  by  the  weightings  for  each  medium.  Weighted  ratings  appear  to 
provide  the  most  effective  basis  for  decisions  since  some  criteria  may  be  much  more 
important  to  the  user  than  are  others. 

Decision  aids  that  are  to  be  used  by  military  personnel  who  may  not  be  highly 
experienced  in  instructional  systems  development  should  provide  strong  user  guidance. 
Examples  of  aids  with  strong  user  guidance  for  establishing  training  situations/levels  are 
provided  by  Pieper  et  al.  (1978)  for  initial  selection  of  training  sites,  Collins  et  al.  (1987) 
for  cost  comparisons  of  training  sites,  and  the  Army  (User's  Manual  for  Predicting  Task 
Retention,  1985)  for  determining  refresher  training  periods.  Examples  of  media  selection 
aids  with  strong  user  guidance  are  provided  by  Reiser  and  Gagne  (1982)  in  a  flowchart 
format  and  Wagner  (1965)  in  a  matrix  format.  Braby  et  al.  (1975)  provides  strong  user 
guidance  for  cost  analysis  and  the  Air  Force  (Nonpersonnel  Studies  and  Analysis  Services 


for  Assessment  of  New  Training  Technologies.  1985)  for  comparing  cost  effectiveness 
ratings. 

Some  of  the  guidance  provided  by  media  selection  aids  appears  to  obstruct  rai'ier 
than  aid  the  decision  process.  This  includes  the  identification  or  discussion  of  criteria 
that  are  not  linked  to  the  decision  process  and  inconsistent  recommendations  for  simi  ,x 
training  conditions.  Such  guidance  can  only  confuse  the  user. 

Training  Units/Task  Types 

As  defined  in  this  report,  the  training  unit  is  the  portion  of  the  overall  instruction  to 
which  a  particular  medium  or  set  of  media  is  applied.  The  training  decision  aids  reviewed 
in  the  present  report  differ  considerably  in  how  they  characterize  a  unit  of  training. 
Some  (e.g.,  Kribs  et  al.,  1983)  specify  a  task  as  a  unit  to  be  trained.  Others  (e.g.,  Merrill 
&  Goodman,  1972)  require  each  task  to  be  analyzed  into  lists  of  training  objectives. 
Collins  et  al.  (1987)  require  consideration  of  combinations  of  related  tasks.  For  many,  the 
unit  of  training  is  unclear. 

Those  aids  that  do  not  base  the  determination  of  training  situations/levels  and  media 
on  learning  behaviors  generally  base  such  decisions  on  the  overall  characteristics  of  the 
task.  A  variation  of  this  approach  that  might  prove  useful  is  to  categorize  tasks  into 
functional  areas  or  tasks  types  (e.g.,  electronics  maintenance).  Then,  separate  decision 
aids  for  each  type  of  task,  each  with  its  own  criteria,  could  be  constructed.  Training 
decision  aids  based  on  specific  task  types  would  have  a  more  limited  set  of  training 
situation/level  and  media  possibilities  than  would  aids  designed  to  cover  all  types  of  tasks. 
Therefore,  this  approach  should  reduce  the  number  of  possibilities  that  must  be  considered 
in  choosing  training  situations/levels  or  media.  Such  reductions  should,  in  turn,  make  the 
decision  process  more  efficient. 
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None  of  the  reviewed  aids  use  functional  areas  or  task  types  as  the  unit  of  training. 
However,  Braby  et  al.  (1975)  provide  specialized  media  selection  models  for  different 
types  of  learning.  Conceivably,  similar  types  of  specialized  media  selection  models  could 
be  constructed  for  different  functional  areas  or  task  types. 

Training  Situation/Level  Criteria 

The  training  situation/level  needs  to  be  considered  both  in  its  own  right  and  as  one  of 
the  variables  that  will  influence  the  selection  of  a  training  method  and  media.  As  used  in 
this  report,  training  situation/level  specifies  who  is  to  receive  the  training,  where  the 
training  is  to  take  place,  and  when  the  training  is  to  occur. 

Who  receives  the  training  is  typically  specified  in  terms  of  occupation,  grade  level, 
and,  for  maintenance  courses,  the  commands  responsible  for  the  task.  Where  the  training 
is  to  take  place  may  be  specified  in  terms  of  formal  school  or  on-the-job  training  (COT). 
When  the  training  is  to  take  place  may  be  specified  in  terms  of  preliminary,  advanced,  or 
refresher  training. 

In  comparison  to  the  rich  body  of  literature  on  the  selection  of  training  media  and  the 
great  varietv  of  media  selection  decision  aids  that  have  been  developed,  little  has  been 
written  concerning  the  determination  of  training  situations/levels.  An  Army  decision  aid 
developed  by  Pieper  et  al.  (1978)  considers  factors  related  to  the  training  situation/level 
along  with  media  considerations.  The  Air  Force  Human  Resources  Laboratory  has 
developed  the  Training  Decisions  System  (TDS)  (Collins  et  al.,  1987),  a  computerized 
system  for  aiding  decisions  concerning  where  personnel  should  receive  training.  The  Army 
has  also  developed  the  User's  Manual  for  Predicting  Military  Task  Retention  U 985),  which 
provides  a  basis  for  determining  when  refresher  training  is  needed.  The  fol.  swing  factors 
and  criteria  have  been  considered  in  determining  the  training  situation/level. 

Who  is  to  be  Trained.  All  of  the  training  situation/level  selection  aids  can  be 
oriented  to  specific  military  occupations,  grade  levels,  or  command  assignments. 

When  Training  Takes  Place.  Pieper  et  al.  (1978)  provide  criteria  for  establishing 
when  initial  training  is  to  take  place.  One  of  these  criteria  is  when  the  task  will  be 
performed,  which,  in  turn,  will  depend  on  grade  level  and  job  responsibilities.  Pieper  et 
al.  also  indicate  that  tasks  should  not  be  trained  until  trainees  have  completed  all 
prerequisite  tasks. 

The  Army's  User's  Manual  for  Predicting  Military  Task  Retention  provides  procedures 
that  appear  to  be  useful  for  determining  refresher  training  periods.  Retention  scores  are 
developed  for  each  task  based  on  user  evaluations  of  task  characteristics.  Tables  in  the 
manual  enable  the  user  to  determine,  for  a  given  retention  score,  how  soon  refresher 
training  will  be  required  in  order  to  maintain  a  specified  percentage  of  personnel  who  are 
proficient  in  the  task.  An  Army  representative  has  indicated  that  a  computerized  version 
of  these  procedures  is  also  available. 

Where  Training  Takes  Place.  In  determining  where  training  is  to  take  place,  Pieper 
et  al.  (1978)  consider  three  options:  initial  training  through  resident  courses  (formal 
schooling),  OJT,  or  no  training.  The  Air  Force's  TDS  considers  similar  options  as  well  as 
field  training  detachments,  career  development  courses,  major  command  programs, 
mobile  training  teams,  contractor  training,  and  interservice  programs  and  courses  (Collins 
et  al.,  1987).  All  of  these  training  options  appear  relevant. 


Task  Priorities  and  Training  Time.  In  determining  where  the  task  should  be  trained, 
Pieper  et  al.  were  primarily  concerned  with  the  overall  amount  of  time  available  for 
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formal  school  training  and  OJT,  the  amount  of  training  time  required  for  each  individual 
task,  and  the  priority  of  the  task.  They  assign  the  highest  priority  tasks  to  formal  school 
training  until  the  overall  training  time  limit  is  exceeded.  The  next  highest  priority  tasks 
are  assigned  to  OJT  until  that  overall  time  limit  is  exceeded.  The  remainder  of  the  tasks 
are  not  designated  for  training. 

Training  priorities  cited  by  Pieper  et  al.  (1978)  are  based  on  a  complex  rating  system 
that  includes  variables  such  as  pay  grade,  time  to  application,  training  time,  percent  of 
members  who  perform  the  task,  and  percent  of  time  spent  performing  the  task.  Task 
priority  and  training  time,  as  defined  by  Pieper  et  al.,  appear  to  be  appropriate  criteria 
for  an  initial  determination  of  where  training  is  to  take  place. 

Task  Appropriateness.  None  of  the  decision  aids  reviewed  consider  the  role  of  task 
appropriateness  in  determining  training  situations/levels  for  specific  tasks.  Task  ap¬ 
propriateness  is  independent  of  task  priority  ratings  as  discussed  by  Pieper  et  al.  A  task 
can  have  a  high  priority  but  not  be  appropriate  for  formal  school  training  because  of  the 
manner  in  which  it  is  performed.  For  example,  tasks  that  are  to  be  practiced  collectively 
and  involve  large  numbers  of  personnel  (such  as  troop  maneuvers)  are  ill  suited  for 
practice  in  a  formal  school.  Task  appropriateness  should  be  included  as  a  criterion  in 
determining  where  training  should  take  place. 

Training  Costs.  Training  costs  appear  to  be  a  relevant  criterion  in  the  establishment 
of  training  situation  levels.  However,  TDS  (Collins  et  al.,  1987)  is  the  only  aid  that  deals 
directly  with  costs  in  the  establishment  of  training  situations/levels.  (Pieper  et  al.  discuss 
costs  for  training  settings,  but  do  not  provide  a  clear  basis  for  selecting  training  situation 
levels  based  on  costs.)  TDS  is  designed  to  deal  with  Task  Training  Modules  (groups  of 
related  tasks)  rather  than  individual  tasks.  The  TDS  computer  program  allows  the  user  to 
formulate  and  plot  alternative  paths  for  satisfying  the  training  requirements  of  each  Task 
Training  Module  and  vary  the  number  of  pieces  of  equipment  provided  in  the  training 
setting.  It  then  computes  resulting  training  cost  (based  on  previous  cost  data  inputs). 
This  allows  the  user  to  identify  characteristics  of  the  training  situation  that  will  result  in 
the  lowest  training  costs. 

In  terms  of  its  utilization,  the  TDS  appears  more  appropriate  for  determining  costs 
associated  with  selected  training  situations/ievels  than  for  guiding  users  in  the  initial 
establishment  of  training  situation/levels.  Therefore,  it  should  be  used  as  an  adjunct  to  a 
training  situation/level  selection  aid.  TDS  should  prove  particularly  helpful  in  comparing 
the  cost  effectiveness  of  potential  training  situations  and  in  justifying  selections  of 
training  situations  and  levels. 

Different  Locations.  An  important  consideration  that  few  decisions  aids  have  dealt 
with  is  whether  a  task  is  to  be  learned,  practiced,  and  evaluated  at  one  or  several 
locations.  For  some  tasks,  it  may  be  desirable  to  introduce  the  task  and  teach  the 
enabling  objectives  (basic  skills  and  knowledge)  in  a  formal  school  setting,  but  to  train  the 
terminal  objective  (involving  practice  of  the  actual  task  performance)  and  evaluate 
performance  in  an  OJT  field  setting.  If  a  school  is  assigned  complete  responsibility  for  a 
task,  but  only  addresses  the  enabling  objectives,  the  terminal  behavior  may  not  be 
properly  trained  and  evaluated.  Therefore,  in  assigning  task  responsibilities  to  formal 
schools  or  OJT,  it  is  desirable  to  provide  settings  for  both  terminal  and  enabling 
objectives. 

Incorporation  of  Existing  Situation/Level  Decision  Aids.  None  of  the  three  aids 


discussed  above  provide  a  completely  satisfactory  basis  for  establishing  training  situa¬ 
tions/levels.  Pieper  et  al.  (1978)  gave  inadequate  consideration  to  costs,  task  ap¬ 
propriateness,  and  possibilities  for  combining  formal  school  training  and  OJT.  Their 
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procedures  are  complex  and  difficult  to  follow.  The  Air  Force's  TDS  (Collins  et  al.,  1987) 
and  the  Army's  User's  Manual  for  Predicting  Military  Task  Retention  (1985)  only  address  a 
few  of  the  issues  involved  in  determining  training  situations/levels.  However,  TDS 
provides  efficient  automated  procedures  for  making  cost  effectiveness  comparisons 
among  different  training  situations/levels  and  the  Army's  manual  provides  easy  to  follow 
procedures  for  determining  refresher  training  intervals.  The  procedures  used  in  these  aids 
could  be  effectively  incorporated  into  a  more  comprehensive  aid  that  addresses  all  of  the 
issues  involved  in  determining  training  situations/levels. 

Training  Presentation  Method/Media  Criteria 

Methods  versus  Media.  Media  selection  aids  vary  greatly  in  the  selection  criteria 
that  they  employ.  One  of  the  major  issues  in  establishing  criteria  for  media  selection  is 
whether  to  establish  separate  criteria  for  training  presentation  methods  and  training 
media.  A  few  of  the  media  selection  aids  reviewed  in  this  report  distinguish  between 
method  and  media  (The  Marine  Corps  Systems  Approach  to  Training  User's  Guide  (Draft), 
1987;  Methods/Media  Selection  Guidelines,  1977;  and  Pieper,  et  al?,  1978).  Similar  kinds 
of  distinctions  were  made  by  Tosti  and  Ball  (1969)  between  medium  and  "presentation 
form"  and  by  Levie  (1975)  between  media  and  media  attributes.  How  can  training 
methods  be  distinguished  from  media?  The  Methods/Media  Selection  Guidelines  define  a 
method  as  a  procedure  or  process  for  attaining  an  objective  and  a  medium  as  a  channel  of 
communication  that  provides  the  hardware  and  software  for  conveying  information.  This 
distinction  between  methods  and  media  is  maintained  in  the  present  discussion  of  media 
selection  criteria. 

The  importance  of  distinguishing  between  methods  and  media  has  been  supported  by 
Clark  (1983).  Following  a  review  of  research  studies  that  compared  the  benefits  of 
different  types  of  media  delivery  systems,  Clark  concluded  that  "Consistent  evidence  is 
found  for  the  generalization  that  there  are  no  learning  benefits  to  be  gained  from 
employing  any  specific  medium  to  deliver  instruction"  (p.  445).  Clark  contends  that 
beneficial  effects  ascribed  to  media  can,  in  actuality,  be  attributed  to  the  learning 
methods  used  in  presenting  the  instruction.  Clark  admits  that  certain  combinations  of 
training  methods  and  media  appear  to  be  more  effective  than  others.  He  points  out,  for 
example,  that  training  approaches  such  as  the  personalized  system  of  instruction  (Keller, 
1968)  and  programmed  instruction  have  proven  to  be  more  effective  than  conventional 
forms  of  instruction.  However,  he  argues  that,  in  both  cases,  the  presentation  methods 
(addition  of  structure,  shorter  steps,  reduced  verbal  loads,  and  self-pacing)  rather  than  the 
media  make  the  difference. 

Orlansky  and  String  (1981)  support  Clark's  contention.  They  reviewed  a  series  of 
military  studies  which  examine  the  effects  of  replacing  conventional,  group-paced 
instruction  with  computer-based  instruction  (CBI),  computer-managed  instruction  (CMI), 
or  individualized  (self-paced)  instruction  without  computers.  They  point  out  that  the 
magnitude  of  reductions  in  training  time  achieved  with  CBI  or  CMI  is  no  greater  than  that 
achieved  with  individualized  instruction.  These  results  suggest  that  the  method  of  self¬ 
pacing  could  account  for  the  efficiency  effects  achieved  with  computer  media. 

Petkovich  and  Tennyson  (1984),  in  a  critical  analysis  of  Clark's  study,  conclude  that 
methods  may  dictate,  to  some  extent,  the  type  of  media  employed.  They  argue,  for 
example,  that  the  complex  mathematical  models  used  by  some  computer  training  systems 
would  be  difficult  or  impossible  for  a  teacher  to  employ  during  the  course  of  instruction. 
It  is  not  difficult  to  visualize  other  examples.  A  conventional  movie  film  would  be  a  poor 
choice  of  medium  for  programmed  instruction  since  it  is  difficult  to  provide  branching 
based  on  student  responses.  The  suitability  of  media  for  methods  is  a  particularly 


important  consideration  for  military  training  involving  weapon  systems  and  simulations  of 
combat  conditions.  For  example,  training  with  a  laser  rifle  allows  for  a  type  of  realistic, 
interactive  practice  that  simply  would  not  be  possible  with  an  actual  weapon. 

These  examples  are  not  in  conflict  with  Clark's  contention  that  the  method  rather 
than  the  medium  is  the  primary  factor  in  determining  instructional  effectiveness.  They 
merely  illustrate  that  the  selection  of  a  method  may  greatly  limit  the  range  of  possible 
media  choices. 

Even  where  different  media  produce  equally  effective  training,  there  still  may  be 
compelling  reasons  to  select  one  medium  over  another.  Costs  are  an  obvious  reason. 
Some  media  are  certainly  more  expensive  to  develop  and  use  than  are  others.  Ease  of 
updating  might  be  another  significant  factor.  Some  media  presentations  are  difficult  to 
change  and  update  while  others  can  be  easily  modified.  Mobility  is  still  another  factor  to 
consider.  Some  media  are  relatively  easy  to  move  from  one  location  to  another  while 
others  are  difficult  to  transport.  Mobility  is  an  important  consideration  for  training 
organizations  such  as  military  mobile  training  teams  that  must  move  from  one  work  site 
to  another  in  order  to  provide  training. 

The  majority  of  media  selection  aids  do  not  distinguish  between  training  methods  and 
media.  Some  appear  to  equate  the  two.  For  example,  the  Air  Force's  Nonpersonnel 
Studies  and  Analysis  Services  (1985),  Courseware's  Media  Selection  (1974),  and  Reiser  and 
Gagne's  model  (1982V  refer  to  a  simulation  as  a  type  of  medium.  A  simulation,  however, 
can  be  viewed  as  a  method  that  is  distinct  from  any  media  that  are  used  in  its 
presentation.  There  are  paper  and  pencil  simulations,  computer  simulations,  mockup 
simulations,  etc. 

Since  different  criteria  may  apply  in  selecting  training  methods  and  media  and  since 
selection  processes  for  methods  and  media  are  often  confounded,  selection  criteria  for  the 
two  should  be  considered  independently.  This  is  done  in  the  following  discussion. 

Types  of  Methods.  Those  reviewed  media  selection  aids  that  separate  methods  from 
media  vary  considerably  in  the  types  of  characteristics  they  consider.  The  Methods/Media 
Selection  Guidelines  (1977)  identify  8  methods:  Lecture,  Demonstration,  Discussion, 
Performance,  Independent  Study,  Programmed  Instruction,  Peer  Training,  and  Simulation. 
Pieper  et  al.  (1978)  list  15  methods:  Case  Study,  Computer  Assisted  Instruction, 
Demonstration,  Games,  Group  Interview,  Guided  Discussion,  In-basket  Exercise,  Peer 
Tutor,  Programmed  Instruction,  Programmed  Practical  Exercise,  Role  Playing,  Study 
Assignment  Book,  Traditional  Classroom,  Traditional  Practical  Exercises,  and  Tutoring. 
While  Tosti  and  Ball  (1969)  and  Levie  (1975)  do  not  identify  training  methods  directly,  the 
factors  they  do  identify  are  suggestive  of  training  methods.  Tosti  and  Ball  list  3  types  of 
presentation  formats:  Stimulus  Factors  (use  of  words,  pictures,  symbols,  motion,  etc.), 
Response  Factors  (including  the  type  and  frequency  of  learner  responses),  and  Manage¬ 
ment  Factors  (involving  actions  to  modify  the  instruction).  Levie  lists  5  types  of  media 
attributes:  Sign  Vehicle  Characteristics  (e.g.,  words  vs.  pictures),  Realism  Cues  (e.g., 
color  vs.  black  and  white  and  motion  vs.  still),  Sensory  Modalities  (e.g.,  seeing  vs. 
hearing  vs.  seeing  and  hearing),  Locus  of  Control  Characteristics  (e.g.,  fixed  vs.  flexible 
pace),  and  Response/Feedback  Characteristics  (e.g.,  overt  vs.  covert  responding). 

A  summary  of  the  various  training  method  characteristics  listed  or  implied  above,  or 
recommended  in  other  media  selection  aids  or  in  this  report  is  presented  in  Table  1. 
These  characteristics  are  categorized  to  clarify  the  types  of  methodological  decisions  to 
be  made.  Listed  method  characteristics  vary  along  a  variety  of  dimensions,  each  of  which 


Table  1 


Categories  of  Training  Method  Characteristics 


Dimension 

Class 

Type 

Control  of  presentation 

Uncontrolled 

Nonprogrammed  independent  study 

Live  control 

Instructor  presentation,  or  peer 
presentation 

Device  control 

Programmed  instruction  (computer  or 
text) 

Control  of  response 

Uncontrolled 

No  response,  nonmonitored  response 

Live  control 

Instructor  controlled,  peer  con¬ 
trolled 

Device  control 

Programmed  instruction  (automated 
or  manual) 

Dispersion  of  training 

Dispersed 

Individuals,  groups 

Face-to-face 

Individuals,  groups 

Fidelity  of  training 

Nonequipment 

Real  performance,  simulated  per- 

conditions 

tasks 

formance 

Tasks  trained  on 

Real  performance,  simulated  per- 

real  equip¬ 
ment 

formance 

Tasks  trained  on 

Real  performance,  simulated  per- 

simulated 

equipment 

formance 

Focus  of  presentation 

Large  group 

Team  training,  independent  training 

Small  group 

Team  training,  independent  training 

Individual 

Independent  training 

Motivation  enhancement 

Gaming 

Role  playing,  computer  or  board 
games,  etc. 

Rewards 

Intrinsic  /extrinsic 

Competition 

Between  individuals,  between  groups 

Passive/active  student 

Passive 

Lecture  or  demonstration 

role 

Active 

Role  playing,  discussions,  etc. 

Provision  of  feedback 

None 

No  quiz  or  test,  quiz  or  test  without 
results 

Provided 

Live  or  by  a  device 

Retention  enhanement 

Memory  develo- 
ment 

Repetition,  mnemonics 

Memory  aid 

3ob  performance  aid 

Stimulus  characteristics 

Audio 

Speech  or  other  sounds,  no  sounds 

Color 

Color,  black  and  white 

Motion 

Full,  partial,  none 

Kinetic 

Balance,  position,  etc. 

Smell 

Odors 

Taste 

Tastes 

Touch 

Textures 

Visuals 

Print,  pictorial,  graphic 
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may  characterize  several  classes  or  types  of  media.  Since  a  single  training  method  may 
be  characterized  on  a  number  of  dimensions  (e.g.,  face-to-face  training  for  a  small  group 
using  programmed  instruction  with  gaming  and  colored  visuals  presented  without  motion), 
the  number  of  possible  method  combinations  that  could  be  identified  from  the  character¬ 
istics  is  very  large.  The  availability  of  such  a  comprehensive  listing  of  method 
characteristics  should  encourage  decision  makers  to  consider  the  full  range  of  methodo¬ 
logical  possibilities  in  designing  training  presentations. 

Method  Selection  Outcomes.  Not  only  are  a  wide  variety  of  methods  available,  the 
outcome  of  training  method  selection  may  be  several  different  methods  rather  than  a 
single  method.  In  learning  to  perform  a  given  task,  different  methods  may  be  appropriate 
for  different  instructional  events.  One  method  might  be  appropriate  for  introducing  a 
task,  another  for  providing  practice,  etc.  In  addition,  many  methods  are  not  mutually 
exclusive  but  can  be  used  together  for  a  given  instructional  event.  For  example,  in 
learning  a  task,  programming  may  be  an  effective  method  for  controlling  student 
responses  and  providing  feedback  while,  at  the  same  time,  repetition  with  gaming  may  be 
an  effective  method  for  helping  the  students  memorize  key  steps.  However,  only  a  few  of 
the  reviewed  media  selection  aids  provide  guidance  for  combining  training  methods. 

Method  Criteria.  While  only  a  few  media  selection  aids  specify  separate  criteria  for 
selection  of  methods  per  se,  some  provide  criteria  for  media  selection  that  carry 
implications  for  method  selection.  These  and  other  criteria  that  are  relevant  to  method 
selection  are  discussed  below. 

1.  Learner  characteristics.  Reiser  and  Gagne  (1982)  point  out  that  some  decision 
aids  use  learner  characteristics  such  as  age  and  reading  ability  as  criteria  in  selecting 
media.  The  importance  of  selecting  media  to  correspond  to  learner  characteristics  has 
been  largely  discounted  by  Clark  (1983).  Learner  characteristics  could  be  important  in 
selecting  training  methods.  For  example,  the  representation  of  printed  text  is  inappro¬ 
priate  for  blind  students  or  nonreaders  whether  the  medium  is  a  book,  film,  or  computer. 
However,  most  of  the  learner  characteristics  cited  in  training  decision  aids  would 
typically  not  occur  in  most  military  settings  because  of  military  entry  requirements  for 
physical  and  mental  abilities. 

2.  Training  situation.  Other  criteria  that  have  been  considered  in  selecting  a 
training  method  relate  to  the  training  situation.  One  important  characteristic  of  the 
training  situation  is  location.  For  example,  Anderson  (1983)  provides  an  option  for 
dispersed  training  in  settings  where  face-to-face  training  is  impractical.  In  such 
situations,  some  independent  method  of  study  would  have  to  be  provided.  Another 
location  consideration  is  whether  the  training  is  to  be  conducted  in  a  classroom  or  in  the 
field.  Obviously,  many  classroom  training  methods  are  inappropriate  for  field  training 
conditions. 

Another  important  characteristic  of  the  training  situation  is  team  performance 
requirements.  If  only  individual  performance  is  involved,  tasks  may  be  practiced 
individually  or  in  groups.  Team  performance  requirements,  however,  may  necessitate 
group  practice. 

Still  another  important  characteristic  of  the  training  situation  is  group  size  (e.g., 
Pieper  et  al.,  1978).  Group  size  may  limit  the  types  of  training  methods  that  can  be 
successfully  employed.  For  example,  at  the  Navy  propulsion  school  at  Great  Lakes, 
students  are  given  watchstation  practice  on  operating  propulsion  equipment.  However, 
because  of  training  time  limitations,  relatively  large  groups  of  students  are  assigned  to 
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each  watchstation.  These  watchstation  tasks  cannot  be  performed  realistically  by  groups 
of  individuals.  Therefore,  in  this  situation,  class  size  inhibits  the  effectiveness  of  the 
training. 

3.  Modes  or  types  of  learning.  Many  training  decision  aids  have  considered  modes 
of  learning  laflective,  cognitive,  and  motor)  or  types  of  learning  (e.g.,  decision  making  or 
gross  motor  skills)  in  selecting  media.  Most  of  the  reviewed  aids  require  task  objectives 
to  be  categorized  by  mode  or  task  type.  Several  aids  (e.g.,  Anderson,  1983;  Braby  et  al., 
1975;  Reiser  5c  Gagne,  1983)  base  media  decisions  on  modes  or  types  of  learning.  Few 
tasks,  however,  involve  only  one  mode  or  type  of  learning.  Most  require  a  mix. 
Furthermore,  labeling  a  task  as  a  "cognitive  task"  or  a  "gross  motor  skill"  does  little  to 
clarify  the  types  of  training  presentation  methods  or  media  that  would  be  appropriate.  In 
media  selection  aids  that  use  modes  or  types  of  learning  in  selecting  media,  the  basis  for 
associating  a  given  media  with  a  given  mode  or  type  of  learning  is  typically  unclear  and 
unstated.  In  general,  most  training  presentation  methods  and  media  can  be  applied  to  a 
broad  range  of  learning  modes  and  types.  The  following  criteria  appear  to  provide  more 
obvious  links  to  specific  training  presentation  methods  and  media. 

4.  Events  of  instruction.  Events  of  instruction  are  of  primary  importance  when  the 
nature  of  the  instruction  would  be  expected  to  change  as  the  course  progresses.  For 
example,  in  training  personnel  to  perform  a  task,  the  instructor  may  introduce  the  task, 
provide  information  about  the  task,  allow  the  trainee  to  practice  the  task,  and,  finally, 
evaluate  the  trainee's  performance. 

Events  of  instruction  have  been  considered  in  a  number  of  media  selection  aids 
and  reviews.  According  to  Reiser  and  Gagne  (1982),  "Most  media  selection  models 
indicate  that  events  of  instruction  should  be  planned  before  selecting  the  media.  .  .  . 
Media  are  then  chosen  based  in  part  on  their  ability  to  present  instructional  events"  (p. 
507). 

As  with  several  previous  cited  criteria,  instructional  events  appear  to  have  the 
most  direct  impact  on  the  selection  of  training  methods  rather  than  media.  For  example, 
a  programmed  method  of  instruction  may  be  appropriate  for  presenting  information  about 
the  task  but  not  for  providing  task  practice.  Once  the  method  is  established,  media  can 
be  selected  from  those  that  are  compatible  with  the  established  method.  As  Reiser  and 
Gagne  point  out,  some  events  of  instruction  can  be  presented  equally  well  by  any  media 
while  others  are  impossible  for  some  media  to  present. 

Despite  the  widespread  recognition  of  the  importance  of  instructional  events  in 
media  selection,  none  of  the  reviewed  media  selection  aids  provide  specific  criteria  for 
selecting  training  methods  based  on  instructional  events.  This  is  unfortunate  since 
different  training  methods  appear  appropriate  for  different  instructional  events  and  some 
criteria  for  selecting  training  methods  only  apply  to  certain  instructional  events.  In  the 
remainder  of  this  section,  each  of  the  selection  criteria  will  be  considered  with  respect  to 
the  instructional  events  to  which  they  apply. 

5.  Stimulus  characteristics.  Stimulus  characteristics  (e.g.,  color,  motion,  audio, 
etc.)  must  be  considered  for  all  instructional  events.  The  selected  training  method  must 
incorporate  all  of  the  stimulus  characteristics  required  to  demonstrate  the  task  or  depict 
critical  task  characteristics.  Media  can  then  be  selected  from  those  that  are  capable  of 
presenting  the  required  stimulus  characteristics.  Although  none  of  the  reviewed  aids 
relate  stimulus  features  directly  to  selection  of  training,  most  relate  stimulus  character¬ 
istics  to  media  selection  (e.g.,  Kemp,  1980). 
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6.  Motivational  requirements.  The  affective  mode  of  learning  deals  primarily  with 
motivational  issues.  However,  a  motivational  component  may  be  involved  in  any  type  of 
learning.  In  building  motivation,  most  training  decision  aids  limit  their  considerations  to 
the  selection  of  a  training  method  that  would  interest  the  students.  Gaming  techniques 
have  been  identified  as  a  training  method  in  some  decision  aids  (e.g.,  Pieper  et  al.,  1978). 
Gaming  techniques  increase  motivation,  especially  during  instructional  events  in  which 
learning  and  practice  require  extensive  repetition. 

Group  practice  can  be  used  to  build  motivation  although  few  aids  have  explicitly 
considered  such  methods.  Group  practice  may  be  desirable  (even  when  team  training  is 
not  required)  to  promote  competition  or  enable  trainees  to  develop  confidence  in 
themselves  and  in  the  abilities  of  their  team  members.  This  would  be  especially 
important  for  training  in  combat  tasks. 

Motivational  factors  apply  to  all  stages  of  learning  with  the  possible  exception  of 
evaluation.  However,  the  focus  of  the  motivation  may  vary.  In  the  introductory  stages  of 
instruction  where  an  overview  of  the  course  is  being  provided,  the  main  motivational 
consideration  is  development  of  an  interest  in  learning  the  task.  In  learning  and 
practicing  the  task,  gaming  techniques  are  relevant.  Group  performance,  as  a  method  for 
enhancing  motivation,  is  relevant,  of  course,  only  to  task  practice. 

7.  Retention  requirements.  Retention  is  one  type  of  cognitive  learning  skill  that  is 
relevant  for  selecting  a  training  method.  If  retention  requirements  are  demanding,  some 
method  for  facilitating  recall  (such  as  use  of  mnemonics,  repetition,  or  job  performance 
aids)  should  be  used.  None  of  the  reviewed  aids  suggest  specific  methods  for  dealing  with 
retention  requirements. 

8.  Programming.  For  assisting  a  student  to  learn  a  task,  a  group  of  related  training 
techniques  may  be  characterized  as  programming.  Definitions  of  programming  vary 
greatly.  However,  for  the  purpose  of  this  report,  programmed  techniques  will  include 
individualization,  self-pacing,  presentation  of  information  in  small  frames  or  units, 
eliciting  student  responses,  and  providing  feedback.  Programming  techniques  have  been 
found  to  improve  training  effectiveness  (Orlansky  &  String,  1981).  However,  few 
guidelines  have  been  provided  to  indicate  where  such  procedures  are  appropriate. 

Programming  can  be  used  to  individualize  instruction  by  directing  students  to 
different  learning  paths  depending  on  their  responses.  For  this  reason,  programming  is 
often  appropriate  where  diverse  student  skills  and  knowledge  are  expected. 

Programming  might  also  facilitate  learning  of  difficult  training  content.  This  is 
because  the  effect  of  programming  is  to  separate  content  into  manageable  training  units 
and  establish  mastery  of  each  unit  before  proceeding  to  the  next. 

While  programming  techniques  can  be  used  wherever  independent  study  is 
needed,  the  need  for  independent  study  is  not,  of  itself,  a  justification  for  programming. 
For  example,  correspondence  courses  may  use  conventional  as  well  as  programmed  texts 
for  providing  independent  study. 

Since  the  development  of  programmed  course  materials  is  apt  to  be  more  costly 
than  the  development  of  conventional  materials,  it  should  probably  be  considered  only 
when  required  by  the  complexity  of  the  instruction  or  the  diversity  of  student  skills  and 
knowledge. 


9.  Automation.  Automation  has  typically  been  considered  as  some  form  of 
computer  application.  Automation  of  training  during  the  learning  and  practice  of  tasks  is 
a  relevant  consideration  whenever  instruction  is  programmed.  Automated  programmed 
instruction  (usually  referred  to  as  computer-assisted  instruction  or  CAI)  has  been  used 
with  a  wide  variety  of  course  work.  With  CAI,  the  student  "interacts  in  real  time,  via  an 
interactive  terminal  with  instructional  material  that  is  stored  in  the  computer"  (Orlansky 
&  String,  1981,  p.  47).  However,  as  stated  earlier,  Orlansky  and  String  also  found  that 
effects  attributed  to  CAI  may,  in  fact,  result  from  other  factors  such  as  self-pacing  of 
instruction. 

Automation  can  have  special  training  advantages  that  go  beyond  the  types  of 
CAI  programming  applications  reviewed  by  Orlansky  and  String.  For  example,  automation 
can  be  an  important  feature  in  developing  games  or  simulations. 

Automation  in  the  form  of  computer-managed  instruction  (CMI)  can  be  con¬ 
sidered  whether  or  not  automation  is  used  in  training  the  task.  Orlansky  and  String  report 
that  CMI,  like  CAI,  does  not  improve  the  efficiency  of  conventional,  group-paced, 
military  training  courses  to  a  greater  extent  than  does  nonautomated  individualized 
instruction. 

However,  automation  with  CMI  offers  advantages  in  addition  to  instructional 
efficiency.  Automated  data  processing  can  facilitate  maintaining  performance  records 
and  relating  these  records  to  other  data.  It  should  be  particularly  effective  in  situations 
where  large  student/instructor  ratios  are  involved  and  where  large  scale  records  of  test 
performance  must  be  maintained  and  compared  to  other  criteria. 

Media  that  involve  automated  methods  of  training  are  often  listed  as  potential 
candidates  by  media  selection  aids.  However,  none  of  the  reviewed  aids  provide  criteria 
for  determining  whether  automated  training  and/or  data  processing  are  required. 

10.  Simulation.  In  providing  task  practice  and  in  evaluating  performance,  there  is 
always  a  question  whether  the  actual  task  is  to  be  performed  or  some  form  of  simulation 
is  to  be  used.  The  reviewed  media  selection  aids  provide  little  guidance  for  determining 
when  a  simulation  should  be  used  and  none  for  determining  what  method  of  simulation 
should  be  employed.  There  are  a  number  of  criteria  to  consider  in  determining  what 
aspects  of  a  task,  if  any,  are  to  be  simulated  during  training. 

Having  the  individual  perform  the  actual  task  has  the  advantage  of  direct 
transfer  of  training.  Personnel  are  trained  and  evaluated  in  performing  exactly  the  same 
behaviors  they  perform  on  the  job.  Task  practice  and  performance  evaluation  should  be 
conducted  under  actual  task  conditions  wherever  it  is  possible  and  practical  to  do  so. 

Simulation  methods  should  typically  be  considered  when  it  is  not  possible  or 
practical  to  perform  the  actual  task.  Simulations  may  be  required  because  tasks  are  too 
dangerous,  time  consuming,  or  costly  to  perform  live;  the  actual  task  conditions  cannot  be 
established  or  controlled;  or  tasks  are  difficult  and  need  to  be  presented  in  a  simplified 
format  in  order  to  facilitate  learning. 

If  the  actual  task  is  to  be  performed,  training  methods  and  media  can  simply 
reflect  task  performance.  However,  when  simulations  are  required,  the  selection  of 
methods  and  media  becomes  more  complicated.  Two  factors  that  should  be  considered  in 
determining  an  appropriate  method  of  simulation  are  the  purpose  of  the  simulation  and 
fidelity  requirements. 


The  purpose  of  the  simulation  will  dictate  whether  the  physical  or  functional 
characteristics  of  the  equipment  must  be  simulated,  or  both.  For  example,  if  the  purpose 
is  to  demonstrate  how  a  system  operates,  the  functional  characteristics  must  be 
simulated.  In  terms  of  the  operation,  the  actual  physical  characteristics  of  the  system 
may  be  incidental.  Conversely,  if  the  purpose  is  to  familiarize  the  student  with  the 
appearance  of  a  system  and  the  position  of  its  parts,  the  physical  characteristics  must  be 
simulated.  In  some  cases,  both  physical  and  functional  characteristics  are  important.  For 
example,  for  a  laser  rifle  simulation,  it  may  be  important  to  duplicate  both  the  weight  and 
size  of  the  actual  weapon  as  well  as  its  aiming  and  firing  functions. 

In  simulating  functional  characteristics  of  a  task,  both  the  functions  of  the 
equipment  and  those  of  the  performer  must  be  considered.  A  simulated  task  may  involve 
actual  behaviors  with  simulated  equipment,  simulated  behaviors  with  actual  equipment,  or 
simulated  behaviors  with  simulated  equipment. 

Having  determined  what  task  characteristics  are  to  be  simulated,  the  fidelity  of 
the  simulation  also  must  be  determined.  In  general,  simulations  should  duplicate  actual 
task  conditions  as  closely  as  possible  to  maximize  transfer  of  training.  However,  as 
Montague  (1982)  points  out,  high  fidelity  may  not  be  necessary  or  even  desirable  in  some 
training  contexts.  For  example,  low  fidelity  may  simplify  task  conditions  in  order  to 
facilitate  learning  during  earlier  stages  of  instruction. 

Montague  also  argues  that,  in  order  to  determine  the  fidelity  requirements  of  a 
simulator,  "it  is  necessary  to  describe  the  task  precisely  in  terms  of  the  way  it  needs  to  be 
represented  mentally"  (p.  9).  This,  in  turn,  requires  a  thorough  analysis  of  learning 
requirements  at  different  levels  of  task  competency.  "What  is  needed  is  a  complete 
description  of  what  individuals  do  at  different  levels  of  competency  and,  then,  more 
adequate  design  of  the  simulation  for  promoting  learning  can  be  contrived"  (p.  8). 
Montague's  statement  implies  that  selecting  an  appropriate  level  of  simulation  fidelity 
requires  an  extensive  analysis  to  determine  the  requirements  of  the  training  situation. 

In  practice,  media  selection  aids  have  given  little  attention  to  simulation 
requirements.  Some  simply  list  simulation  as  a  choice  of  media.  A  few  require  the  user 
to  determine  whether  a  simulation  is  required.  None  provide  criteria  for  determining  the 
type  of  simulation  or  its  fidelity.  However,  it  is  the  characteristics  of  the  simulation  that 
indicate  what  media  characteristics  would  be  necessary  to  present  the  simulation. 

Types  of  Media.  Media  specifications  vary  greatly  from  one  decision  aid  to  another. 
Some  of  the  reviewed  aids  do  not  list  media  candidates  but  require  the  user  to  determine 
the  types  of  media  to  be  considered.  Of  those  that  do  list  media,  some  only  consider  a 
few  media  or  media  categories  while  others  consider  a  large  variety  of  specific  media. 
Table  2  presents  a  summary  of  the  types  of  training  media  that  have  been  considered  in 
media  selection  aids. 

Media  Selection  Outcomes.  Since  it  may  be  desirable  to  use  several  different 
training  methods  with  the  same  task  and  since  different  methods  may  require  or  work  best 
with  different  media,  a  mix  of  media  may  be  appropriate  for  any  given  instructional 
application.  Some  media  selection  decision  aids  have  specifically  provided  for  the 
selection  of  media  mixes  (Braby  et  al.,  1971;  Gagne  &  Briggs,  1974;  Goodman,  1971; 
Holden,  1974;  Kemp,  1980;  Kribs  et  al.,  1983;  Leibium,  1980;  Merrill  &  Goodman,  1972; 
Methods/Media  Selection  Guidelines,  1977;  Reiser  &  Gagne,  1982;  and  Romiszowski, 
1974).  Few,  however,  have  provided  an  adequate  basis  for  determining  the  type  of  media 
mix  that  is  required. 


Table  2 


Types  of  Training  Media 


Audio  Recordings 

Audiodisc  playback  system 
Audiotape  system 

Dial  access  audio  information-retrieval  system  (random  or  scheduled) 
Radio  (active  and  passive) 

Record 

Telephone  (individual  or  conference  system) 

Board  Displays 
Blackboard 
Bulletin  board 
Diorama 
Felt  board 
Magnetic  board 
Whiteboard 

Charts/Posters 

Computers 

Film  Presentations 
Silent  film 
Sound  film 

Filmstrip  Presentations 
Holographs 

Interactive  Videodiscs 
Live  Instructor  Presentations 
Microfilm  Presentations 
Models/Mockups 

Equipment  displays  (2- D  or  3-D,  cutaway  or  whole,  static  or  dynamic) 
Terrain  models  (fixed  models,  sand  tables,  etc.) 

Opaque  Projections 

Realia 

Kits 

Real  equipment 
Real  objects 
Real  people 
Specimens 

Slide  Presentations 
Random  access  slides 
Slide  projection  system 
Sound/slide  projection  system 

Teaching  Machines 
branching 
Linear 

Televised  Presentations 
Broadcast  tv 
Closed  circuit  TV 
Videodisc 
Videotape 

Textual  Presentations 
Handouts  ~ 

Periodicals 
Reference  books 
Study  card  sets 
Textbooks 
Workbooks 


Transparency  Presentations 


Media  Criteria.  The  following  criteria  apply  to  the  selection  of  media  or  combina¬ 
tions  of  media  and  methods. 

1.  Training  method.  The  first  major  consideration  in  selecting  media  is  the  training 
method.  As  Petrovich  and  Tennyson  ( 1 984)  point  out,  in  some  cases  the  method  may 
dictate  the  medium  to  be  used.  In  any  case,  each  medium  must  be  compatible  with  the 
training  methods  employed.  For  example,  if  programmed  instruction  has  been  selected  as 
a  method,  then  a  medium  that  is  compatible  with  a  programmed  approach  (e.g.,  computer 
instruction  or  programmed  text)  should  be  selected.  If  a  high  fidelity  functional 
simulation  method  is  required,  a  medium  that  can  provide  those  features  (such  as  an 
interactive  videodisc)  would  be  required.  If  the  method  calls  for  displays  of  motion  or 
color  or  presentations  of  sound,  then  media  that  have  these  capabilities  should  be 
selected. 

While  this  study  has  argued  that  the  selection  of  media  should  be  based,  in  part, 
on  training  method  decisions,  the  final  determination  of  training  methods  may  have  to  be 
made  in  conjunction  with  media  selection.  This  is  due  to  the  fact  that  the  same  criteria 
that  apply  to  training  media  (costs,  training  time,  etc.)  also  apply  to  training  methods. 
From  this  perspective,  training  method  decisions  made  earlier  must  be  considered  as 
preliminary  and  subject  to  revision  during  the  selection  of  training  media. 


As  previously  stated,  aids  that  provide  procedures  for  selecting  training  presen¬ 
tation  methods  separately  from  training  media  were  developed  by  the  Marine  Corps 
(Marine  Corps  Systems  Approach  to  Training  User's  Guide  (Draft),  1987),  the  Navy 
(Methods/Media  Selection  Guidelines,  1977),  and  the  Army  (Pieper  et  al.,  1978).  However, 
none  of  these  aids  provides  explicit  procedures  for  basing  media  choices  on  the  selected 
training  methods,  as  is  recommended  in  this  report.  Because  of  the  complexities  involved 
in  linking  media  to  methods,  effective  guidance  is  needed. 

2.  Training  costs.  While  Clark  (1983)  concluded  that  media  do  not  affect  training 
effectiveness  when  considered  apart  from  the  training  method  employed,  he  admitted  that 
media  can  impact  on  factors  such  as  training  costs  and  time.  A  number  of  media 
selection  decision  aids  have  considered  training  cost  as  a  criterion.  Some  of  these  aids 
(e.g.,  Nonpersonnel  Studies  and  Analysis  Services,  1985  and  Lonigro  &  Eschenbrenner, 
1973)  have  simply  rated  media  on  a  cost  scale  (e.g.,  low,  medium,  and  high).  Others  (e.g., 
Braby  et  al.,  1975)  have  developed  elaborate  formulas  for  specifying  exact  costs.  These 
formulas  may  include  implementation  costs  as  well  as  initial  development/acquisition 
costs.  Costs  for  different  media  systems  are  often  compared  over  periods  of  time. 

The  Air  Force's  Nonpersonnel  Studies  and  Analysis  Services  media  selection 
model  advocates  the  comparison  of  media  systems  in  terms  of  a  cost-effectiveness 
relationship  designed  by  Keeney  and  Raiffa  (1976).  Selecting  the  most  cost-effective 
medium  is  a  straight  forward  process  as  long  as  one  medium  is  both  less  costly  and  more 
effective  than  the  others.  However,  if  one  medium  is  less  costly  but  another  is  more 
effective,  how  should  the  selection  be  made?  The  best  rule  of  thumb  in  this  situation  may 
be  to  take  the  most  effective  method  that  is  affordable. 

Conducting  a  detailed  cost  analysis  is  a  time  consuming  process  that  requires 
considerable  information  concerning  training  facilities  and  student  throughput  that  may 
not  be  readily  available  to  decision  makers.  Such  cost  analysis  may  only  be  appropriate  in 
situations  where  tradeoffs  between  high  cost  training  media  (e.g.,  actual  equipment  vs. 
simulators)  are  being  considered,  and  the  decision  makers  have  the  necessary  time  and 
information  to  conduct  an  extended  cost  evaluation.  A  media  selection  aid  should  specify 


the  conditions  that  would  necessitate  a  detailed  cost  analysis.  None  of  the  reviewed  aids 
did  that. 

Training  time  is  typically  considered  to  impact  on  training  costs.  However,  it 
can  be  an  important  consideration  in  its  own  right  and  should  be  used  as  a  criterion 
wherever  efficiency  of  training  is  important. 

3.  Revision,  mobility,  and  development  time.  Additional  criteria  that  appear  to  be 
appropriate  for  media  selection  are  ease  of  revisior,  level  of  mobility,  and  development 
time.  If  the  course  content  is  likely  to  require  frequent  modification  and  updating,  then  it 
would  be  desirable  to  choose  media  that  allow  changes  to  be  made  easily.  If  the  course 
work  is  to  be  moved  from  place  to  place,  the  chosen  media  should  be  mobile.  Finally,  if 
course  development  time  is  limited,  media  that  are  readily  acquired  and  allow  for  quick 
development  of  course  work  should  be  chosen. 

Media  requirements  such  as  ease  of  revision,  mobility  of  training  devices,  and 
development  time  are  mentioned  in  several  media  selection  aids.  However,  users  need 
guidance  to  determine  which  media  systems  best  satisfy  these  requirements.  None  of  the 
reviewed  aids  provides  such  guidance. 

4.  Availability  and  personal  preference.  Media  selection  may  be  looked  at  as  a 
process  of  elimination.  The  establishment  of  method,  cost,  revision,  mobility,  and 
development  time  considerations  should  greatly  reduce  the  number  of  potential  media 
that  could  be  used.  For  many  training  conditions,  however,  a  number  of  reasonable 
possibilities  may  still  exist  at  this  point  of  the  analysis.  Final  decisions  may  best  be  made 
by  the  individual  who  is  to  provide  the  instruction.  Some  media  selection  decision  aids 
have  suggested  criteria  that  would  be  appropriate  at  this  level  of  decision  making  (e.g., 
The  Marine  Corps  Systems  Approach  to  Training  User’s  Guide  (Draft),  1987). 

One  criterion  for  the  instructor  would  be  availability.  If  several  media  are 
possible  choices,  the  instructor  may  as  well  work  with  a  medium  that  is  already  available 
rather  than  one  that  must  be  acquired.  This  is  especially  true  if  instructional  packages 
have  already  been  developed  and  are  available  for  the  ins*ructor  to  use.  For  example,  if 
the  instructor  has  a  choice  between  introducing  a  task  wun  a  live  lecture  or  a  film  that 
has  already  been  prepared,  the  use  of  the  film  would  save  lecture  preparation  time. 

Another  criterion  for  the  instructor  that  may  be  considered  at  this  final  level  of 
decision  making  is  familiarity  with  the  media  choices.  When  other  factors  have  been 
equated,  it  may  be  best  to  allow  instructors  to  use  media  with  which  they  can  work 
comfortably. 


CONCLUSIONS 

1.  Most  existing  training  decision  aids  have  not  been  found  to  be  effective  in  aiding 
personnel  to  select  training  situations/levels  or  media. 

2.  Desirable  attributes  for  military  training  decision  aids  can  be  identified  and 
justified  from  reviews  of  current  research  and  examinations  of  existing  aids. 

3.  Existing  decision  aids  for  determining  relative  costs  of  training  paths  and 
refresher  training  requirements  appear  to  be  effective  adjuncts  to  aid  in  the  process  of 
establishing  training  situations/levels. 


k.  None  of  the  reviewed  aids  satisfy  all  of  the  required  attributes  identified  in  this 
report.  However,  many  existing  training  decision  aids  contain  desirable  criteria  that 
could  be  incorporated  into  more  effective  aids  that  would  be  suitable  for  military 
requirements. 

5.  Methods  for  designing  more  effective  military  training  decision  aids  that 
incorporate  desirable  selection  criteria  are  needed. 


RECOMMENDATIONS 

The  following  recommendations  provide  a  basis  for  guiding  the  design  and  develop¬ 
ment  of  effective  military  training  decision  aids  intended  for  use  in  establishing  training 
situations/levels  and  selecting  media.  Military  training  decision  aids,  based  on  these 
recommendations,  should  improve  the  quality  of  training  decisions  and  result  in  more 
cost-effective  training  programs.  Each  recommendation  references  the  specific  discus¬ 
sion  (in  this  report)  on  which  it  is  based. 

1.  Develop  training  decision  aids  for  specific  military  applications  (see  pp.  5  &  6). 

Training  decision  aids  used  by  the  military  should  be  designed  for  the  specific 
applications  that  they  are  to  be  used.  They  should  be  compatible  with  user  abilities  and 
resources  and  the  student  populations  for  which  they  are  designed. 

2.  Combine  computerized  flowchart  and  matrix  formats  in  structuring  decision  aids 
(see  p.  6). 

A  flowchart  with  decision  points  should  be  used  initially  to  reduce  the  choices  to 
a  manageable  number.  Comparisons  of  remaining  choices  should  be  guided  by  a  matrix 
type  format.  A  menu-driven  computer  system  should  be  used  to  guide  the  user  through 
these  decision  processes. 

3.  Provide  detailed  guidance  for  decision  makers  (see  pp.  6  &  7). 

Military  users  may  have  little  experience  in  establishing  training  situa¬ 
tions/levels  and  selecting  media.  Therefore,  they  should  receive  detailed  guidance  in 
terms  of  the  types  of  training  methods  and  media  that  should  be  considered  and  the 
characteristics  of  the  training  situation  that  would  lead  to  the  selection  of  specific 
methods  and  media. 

4.  Develop  decision  aids  for  specific  types  of  tasks  (see  pp.  7  Sc  8). 

Existing  training  decision  aids  are  designed  to  deal  with  a  wide  range  of  tasks  or 
training  objectives.  In  contrast  to  these  approaches,  it  is  recommended  that  training  aids 
be  developed  for  specific  types  of  tasks.  This  approach  will  enable  users  to  deal  with 
similar  tasks  in  the  same  manner  and  also  reduce  the  number  of  decisions  needed  to  select 
training  situations/levels  or  media  for  a  given  type  of  task. 

5.  Select  training  situations/levels  based  on  task  prerequisites,  training  priorities, 
available  training  time,  task  appropriateness,  and  training  costs  (see  pp.  8-10). 

Training  situation/level  specifications  include  who  is  responsible  for  performing 
the  task,  when  the  task  requires  training,  and  where  the  training  should  take  place.  If 
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portions  of  the  task  are  to  be  taught  at  different  locations,  specifications  should  also 
indicate  when  and  where  each  portion  is  to  be  taught.  Criteria  that  sfiould  be  used  in 
specifying  training  situations/levels  are  training  priorities,  available  training  time,  task 
appropriateness,  and  training  costs. 

6.  Use  existing  situation/level  decision  aids  for  determining  relative  costs  and 

refresher  training  intervals  (see  pp.  9-10).  ~~  — — — 

The  decision  process  for  selecting  training  situations/levels  should  include  a 
system  similar  to  the  Air  Force's  computerized  Training  Decisions  System  for  determining 
comparative  costs  of  different  training  locations.  It  should  also  include  a  system  similar 
to  the  Army's  User  Manual  for  Predicting  Military  Task  Retention  for  determining  when 
refresher  training  should  occur. 

7.  Select  training  presentation  methods  based  on  training  location,  group  character¬ 

istics,  training  events,  stimulus  characteristics,  affective  (motivational)  requirements, 
level  of  difficulty,  programming  requirements,  group  practice,  simulation  requirements, 
and  requirements  for  automation  of  testing  and  data  management  (see  pp. 13-17).  ” 

Training  presentation  methods  should  be  established  independently  prior  to  the 
selection  of  training  media.  Training  location  and  group  characteristics  should  be  used 
first  to  reduce  the  number  of  possible  choices.  Then,  specific  training  presentation 
methods  should  be  determined  for  each  event  of  instruction.  In  introducing  a  task  and  in 
all  other  instructional  events,  stimulus  characteristics  (requirements  for  color,  sound 
motion,  etc.  that  are  needed  to  effectively  depict  the  task),  and  affective  (motivational) 
requirements  are  relevant.  Stimulus  characteristics  should  be  based  on  the  sensory 
characteristics  required  for  the  performance  of  the  task,  and  the  development  of  student 
interest.  Motivational  considerations  should  be  based  on  the  need  to  create  interest  or  to 
promote  high  levels  of  performance. 

In  initial  learning,  level  of  difficulty  and  programming  requirements  should  also 
be  considered.  If  coursework  is  difficult  due  to  retention  requirements,  some  method  for 
facilitating  recall  (such  as  mnemonics,  repetition,  or  job  performance  aids)  should  be  used. 
If  extensive  repetition  is  required  or  motivation  is  a  problem,  gaming  techniques  should  be 
considered. 

If  coursework  is  difficult  due  to  the  complexity  of  the  material,  programming 
should  be  considered.  Programmed  instruction  provides  control  over  student  progress, 
elicits  student  responses,  and  provides  feedback.  Programmed  instruction  may  be  more 
costly  to  develop  than  conventional  instruction  but  it  has  frequently  been  found  to 
improve  learning  effectiveness. 

Automated  instruction  using  computer  presentations  may  be  considered  for  use 
in  conjunction  with  programmed  instruction  or  gaming  techniques. 

In  practicing  the  task,  group  practice  requirements  and  simulation  requirements 
should  also  be  considered.  Group  practice  may  be  required  to  train  personnel  to  perform 
as  a  team.  However,  even  when  team  skills  can  be  obtained  through  individual 
instruction,  group  practice  may  still  be  appropriate  for  providing  competition  or  building 
confidence  in  team  members. 


Simulations  should  be  considered  whenever  tasks  are  too  dangerous,  time 
consuming,  costly,  or  difficult  to  train  under  actual  task  conditions.  If  equipment  is 


£,  ■  i , ■  ftv  jg~-  »liv  Ji %  •  .< j** M<  tyv  fctfa*  .v«  »<v j  *j'M  •t 


involved,  it  should  be  determined  if  the  functional  or  physical  characteristics  of  the 
equipment  need  to  be  simulated  or  both.  Class  size  should  also  be  taken  into  account  in 
developing  simulations.  Details  of  the  simulation  method  cannot  be  specified  until  an 
extensive  analysis  of  the  learning  requirements  of  the  students  has  been  conducted. 

In  testing  or  evaluating  performance,  requirements  for  automation  of  testing  and 
data  management  procedures  should  be  considered.  Automation  of  testing  should  be 
considered  for  situations  where  large  populations  of  students  are  involved  and/or  test 
performance  data  must  be  retained  and  compared  against  other  criteria. 

8.  Select  training  media  based  on  training  presentation  methods;  requirements  for 
modification,  mobility,  and  development  time:  costs;  availability;  and  instructor 


preference  (see  p.  19-20 

First,  media  choices  that  are  incompatible  with  selected  training  presentation 
methods  should  be  eliminated.  Then,  other  media  choices  should  be  further  reduced  based 
on  practical  considerations.  If  the  coursework  is  expected  to  undergo  considerable  change 
over  time,  then  media  with  presentations  that  are  difficult  to  modify  should  be 
eliminated.  If  a  high  degree  of  training  mobility  is  required,  media  that  are  not  easily 
transported  should  be  eliminated.  If  a  short  development  time  is  required,  media  that 
require  extensive  time  to  develop  should  probably  not  be  considered.  If  none  of  the 
prospective  media  can  satisfy  all  three  requirements,  then  the  relative  importance  of 
these  requirements  should  be  determined  and  the  media  that  satisfy  the  most  high  priority 
requirements  should  be  selected. 

When  more  than  one  acceptable  candidate  media  have  been  identified,  cost- 
effectiveness  considerations  should  be  applied.  A  good  rule  of  thumb  is  to  determine  the 
relative  effectiveness  of  media  combinations  and  then  select  the  most  effective  mecia 
combination  that  can  be  afforded.  Effectiveness  can  be  judged  in  terms  of  how  well  the 
media  present  the  method  and  satisfy  requirements  for  modification,  mobility,  and 
available  development  time.  Effectiveness  judgments  should  be  based  on  weighted  ratings 
of  significant  characteristics. 

Initial  cost  judgments  should  be  made  with  the  assistance  of  a  media/cost  matrix 
that  indicates  relative  levels  of  media  cost  (high  to  low).  For  medium  or  low  cost  media, 
detailed  cost  analyses  are  probably  not  cost-effective.  However,  where  two  or  more  high 
cost  media  are  being  considered,  elaborate  cost  comparisons  may  be  justified.  In  making 
specific  cost  comparisons,  it  is  important  to  take  into  account  whether  existing  media 
presentations  are  already  available.  Costs  of  previously  acquired  media  systems  should  be 
listed  as  zero  unless  additional  costs  are  involved  for  maintenance. 

Where  two  or  more  candidate  media  appear  equally  effective  and  affordable,  it 
is  probably  best  to  leave  it  to  the  individual  instructors  to  decide  which  they  prefer  to  use 
based  on  the  availability  of  the  media  and/or  on  their  own  expertise  and  familiarity  with 
the  media. 


9.  Consider  a  combination  of  training 
each  task. 


sresentation  methods  and  media  for  teaching 


A  variety  of  training  presentation  methods  should  be  considered  for  each  task  to 
be  trained  (see  pp.  11-13).  Method  selection  outcomes  should  often  be  mixes  of  methods 
rather  than  a  single  instructional  approach  (e.g.,  programmed  instruction  and  repetition 
might  be  used  together  effectively  in  aiding  the  student  to  learn  the  task)  (see  p.  13). 
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Different  training  presentation  methods  should  be  considered  for  different  instructional 
events  (introduction,  learning,  practice,  and  evaluation)  (see  p.  14).  Also,  since  it  may  be 
desirable  to  use  several  different  training  methods  for  the  same  task,  and  since  different 
methods  may  require  or  work  best  with  different  media,  a  combination  of  media  should  be 
considered  for  any  given  instructional  application  (see  p.  17). 
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GLOSSARY 


Special  terms  used  in  this  report  are  defined  in  relation  to  the  context  of  this  report. 
They  may  be  used  differently  in  other  reports  or  contexts. 

Affective.  A  mode  of  learning  related  to  motivation  or  attitude.  An  affective  mode  of 
learning  might  promote  positive  job  attitudes,  encourage  team  spirit,  etc. 

Cognitive.  A  mode  of  learning  related  to  thinking  skills.  A  cognitive  mode  of  learning 
might  promote  such  skills  as  remembering,  decision  making,  etc. 

Computer-assisted  Instruction  (CAI).  A  training  system  where  a  computer  provides 
instruction  as  well  as  other  training  functions. 

Computer-managed  Instruction  (CMI).  A  training  system  where  a  computer  provides 
training  functions  such  as  feedback,  testing,  and  data  management  but  not  instruction. 

Content  Stimulus  Characteristics.  Physical  characteristics  of  the  task  that  are  part  of 
the  training  content  such  as  color,  sound,  motion,  etc. 

Dispersed  Training.  A  training  method  that  provides  instruction  to  trainees  at  separate 
locations  rather  than  to  a  group  of  trainees  at  the  same  location.  Correspondence 
coursework  and  closed  circuit  television  are  two  methods  commonly  used  to  provide 
dispersed  training. 

Feedback.  Information  provided  to  trainees  concerning  the  adequacy  of  their  responses 
during  training  or  evaluation. 

Flowchart  Format.  A  training  decision  aid  format  that  graphically  displays  sequences  of 
steps  and  often  provides  decision  points  with  alternative  branches. 

Functional  Area.  A  related  group  of  tasks  with  generic  characteristics  suitable  for  a 
particular  set  of  training  situations/levels  and/or  training  media.  The  terms  "functional 
area"  and  "task  type"  are  used  interchangeably  in  this  report. 

Gaming.  A  training  presentation  method  that  presents  instruction  in  the  form  of  a  game 
to  motivate  trainees.  The  gaming  technique  may  simulate  the  actual  behaviors  to  be 
trained  (as  in  war  games)  or  provide  context  that  is  independent  of  the  training  content 
(for  example,  by  associating  response  times  for  mathematical  solutions  with  the  speed  of 
a  graphic  depiction  of  a  running  horse). 

Individualized  Training  or  Instruction.  A  training  presentation  method  that  enables 
individuals  to  proceed  at  their  own  pace,  in  contrast  to  conventional  group-paced  training 
where  a  class  of  individuals  receives  the  same  training  inputs  and  proceeds  at  the  same 
pace.  Individuals  may  receive  different  training  based  on  their  diagnosed  requirements. 

Instructional  Events.  Events  that  occur  during  the  presentation  of  training  programs.  For 
the  present  report,  instructional  events  include  introduction  of  the  task  content,  presenta¬ 
tion  of  information  relevant  to  the  task,  task  practice,  and  evaluation  of  performance. 

Matrix  Format.  A  training  decision  aid  format  that  graphically  relates  two  or  more  types 
of  information  (e.g.,  types  of  training  media  and  types  of  training  methods)  to  provide  an 
overall  depiction  of  relationships  (e.g.,  which  media  are  appropriate  for  which  methods). 
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Mode  of  Learning.  A  general  type  of  learning  that  may  occur  during  training.  The  three 
modes  of  learning  referred  to  in  this  report  are  affective,  cognitive,  and  psychomotor. 

Psychomotor.  A  mode  of  learning  that  involves  physical  skills.  A  psychomotor  mode  of 
learning  might  promote  skills  involving  body  movement,  manual  dexterity,  etc. 

Self-paced.  A  training  presentation  method  that  enables  learners  to  proceed  through 
instruction  at  their  own  pace. 

Simulation.  A  training  presentation  method  in  which  the  physical  and/or  functional 
aspects  of  a  task  are  simplified  or  modified  to  allow  practice  or  facilitate  learning.  A 
mockup  is  a  type  of  simulation. 

Simulation  Fidelity.  The  degree  to  which  a  simulation  replicates  the  actual  physical 
and/or  functional  aspects  of  a  task. 

Stage  of  Training.  The  point  at  which  training  is  provided  in  the  development  of  the 
individual's  task  competency.  !n  the  present  report,  stages  of  training  include  prelimi¬ 
nary,  advanced,  and  refresher  training. 

Stimulus  Characteristics.  Features  of  the  task  that  involve  sensory  inputs  (color,  sound, 
motion,  etc.). 

Task  Type.  A  related  group  of  tasks  with  generic  characteristics  suitable  for  a  particular 
set  of  training  situations/levels  and/or  training  media.  The  terms  "task  type"  and 
"functional  area"  are  used  interchangeably  in  this  report. 

Training  Medium  (Media).  The  channel  of  communication  that  provides  the  hardware  and 
software  fur  conveying  information  as  distinguished  from  the  "method"  or  "method  of 
presentation,"  which  is  a  procedure  or  process  for  obtaining  a  training  objective.  An 
example  of  a  medium  would  be  a  computer. 

Training  Presentation  Method.  The  procedure  or  process  for  obtaining  a  training  objective 
as  distinguished  from  a  "medium,"  which  refers  to  the  channel  of  communication  that 
provides  the  hardware  and  software  for  conveying  information.  An  example  of  a  method 
of  presentation  would  be  programmed  instruction. 

Training  Situation/Level.  The  specification  of  where  training  is  to  be  provided  (in  formal 
schools  or  at  unit  commands),  when  it  is  to  be  provided  (during  preliminary,  advanced,  or 
refresher  training),  and  who  should  receive  the  training  (what  MOSs,  grade  levels,  and, 
where  appropriate,  command  responsibilities). 

Training  Unit.  The  portion  of  the  overall  instruction  to  which  a  particular  medium  or  set 
of  media  is  addressed. 

Type  of  Learning.  The  type  of  outcome  to  be  achieved  for  a  given  task  within  a  given 
mode  of  learning.  For  example,  one  type  of  learning  that  might  be  required  for  a  task 
within  the  cognitive  mode  is  concept  formation.  One  type  of  learning  that  might  be 
required  for  a  task  within  the  psychophysical  mode  is  manual  dexterity. 
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DESCRIPTIONS  OF  TRAINING  DECISION  AIDS 


This  appendix  contains  a  description  of  each  of  the  training  decision  aids  discussed  or 
referenced  in  this  report.  Aids  for  establishing  training  situations/levels  and  for  selecting 
training  media  are  included.  Each  training  decision  aid  is  identified  by  its  author  or  the 
title  of  the  book,  article,  or  publication  in  which  the  aid  is  described. 

The  following  information,  as  appropriate,  is  provided  for  each  training  decision  aid: 

1.  The  intended  users  of  the  aid. 

2.  The  student  population  that  the  aid  was  designed  to  support. 

3.  The  media  and/or  situation/level  covered  by  the  aid. 

4.  The  format  for  providing  user  direction  (flowchart,  matrix,  etc.). 

5.  The  procedures  followed  and  the  criteria  used  in  applying  the  aid. 

6.  The  outcome  of  the  selection  process. 

The  effectiveness  of  the  various  procedures  and  the  appropriateness  of  the  criteria 
employed  by  the  aids  are  discussed  in  the  body  of  this  report.  However,  in  specific 
instances  where  additional  guidance  may  be  needed,  no  further  guidance  is  provided. 


Anderson  (1983) 


Intended  Users 

This  media  selection  decision  aid  is  designed  for  use  by  course  writers  and 
instructors. 

Student  Population 

The  target  population  for  the  instruction  appears  to  be  any  student  audience. 

Media  Covered 


This  aid  covers  10  classes  of  media:  audio,  printed  material,  audio-print,  projected 
still-visual,  audio-projected  still-visual,  motion-visual,  audio-motion  visual,  physical 
objects,  human  and  situational  resources,  and  computers.  Each  clas  >  consists  of  from  one 
to  four  media  for  a  total  of  37  specific  media.  Specific  media  vary  from  audio  tapes, 
manuals,  and  job-aids  to  computers. 

Format  for  Providing  User  Direction 

User  direction  is  provided  by  a  combination  of  flowcharts,  check  lists,  and  text. 
Procedures  and  Criteria 


The  aid  directs  users  to: 

1.  Decide  whether  their  purpose  is  to  provide  "instruction"  or  "information."  (In  the 
present  military  training  context,  we  are  only  interested  in  providing  instruction.)  The 
users  determine  whether  it  is  appropriate  to  use  an  instructor  supported  by  an  instruc¬ 
tional  aid  or  an  instructional  medium  (a  stand-alone  presentation  that  does  not  require  an 
instructor).  Selection  of  an  instructional  medium  is  indicated  if  (a)  the  instruction  must 
be  reproducible  for  a  large  audience,  (b)  it  is  to  be  self-paced,  (c)  the  content  is  locked  in 
(unchangeable),  or  (d)  use  of  face-to-face  instruction  is  impractical. 

Selecting  a  stand-alone  instructional  medium  requires  most  of  the  same  decisions 
as  selecting  instructional  aids  to  support  an  instructor-led  presentation.  However,  the 
same  decisions  may  lead  to  different  classes  of  media. 

t 

2.  Determine  the  required  lesson  characteristics  and  select  an  appropriate  class  of 

media.  The  users  first  determine  whether  the  mode  of  learning  is  affective,  cognitive,  or 

I  psychomotor. 

» 

l 

•  a.  If  the  learning  mode  is  affective,  the  users  are  directed  to  potential  classes 

,  of  media  depending  on  whet  ■sr  the  content  deals  with  interpersonal  skills,  and  whether 

the  display  of  motion  and/or  sound  is  necessary. 

b.  If  the  mode  of  learning  is  cognitive  or  psychomotor,  the  users  must 
determine  whether  the  lesson  involves  objects  or  things  unfamiliar  to  the  student.  If  it 
i  does  not,  the  users  are  directed  to  potential  classes  of  media  depending  on  whether 

[  interpersonal  skills  are  involved,  whether  displays  of  motion  and/or  sound  are  involved, 

|  and  whether  demonstrating  actual  performance  in  class  is  practical  (economical,  safe, 

etc.). 


c.  If  the  lesson  involves  objects  or  activities  unfamiliar  to  the  student,  the 
users  are  directed  to  potential  classes  of  media  depending  on  whether  interpersonal  skills 
are  involved;  whether  displays  of  motion,  sound,  or  color  are  involved;  whether 
demonstrating  the  real  object  or  activity  in  class  is  practical;  and,  if  so,  whether  it  is 
desirable  to  display  exaggerated  views  or  presentations. 

3.  Review  a  list  of  potential  media  for  each  media  class  and  select  an  appropriate 
media  class  and  medium  within  that  class.  No  guidance  is  provided  for  performing  this 
procedure. 

4.  Review  statements  concerning  the  advantages  and  disadvantages  of  the  selected 
medium  and  to  complete  a  checklist  of  items  relevant  to  the  use  of  the  medium,  student 
expectations,  lesson  content,  and  objectives.  However,  checklists  are  not  provided  for  all 
of  the  media  or  media  classes.  Some  of  the  checklists  appear  to  address  issues  already 
dealt  with  in  step  3.  Issues  that  are  considered  (such  as  safety  and  economic  factors)  vary 
greatly  from  one  checklist  to  another. 

Users  are  instructed  to  accept  or  reject  their  initial  choice  based  on  the  results  of 
their  review.  Users  who  reject  their  initial  choice  are  instructed  to  select  one  of  the 
other  media  from  the  appropriate  classes  and  repeat  the  procedure. 

Outcome  of  Selection  Process 

At  the  conclusion  of  the  selection  process,  each  user  has  selected  a  single  medium  for 
use  as  an  instructional  aid  or  as  an  instructional  medium. 
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Boucher  et  al.  (1971) 


Intended  Users 

This  media  selection  decision  aid  is  designed  for  use  by  training  specialists. 
Student  Population 

The  target  population  for  the  instruction  consists  of  Navy  studen's. 


Media  Covered 

This  aid  considers  29  specific  media  types.  These  media  types  include  conventional 
media  as  well  as  simulators,  teaching  machines,  and  a  few  specialized  systems  designed 
for  specific  training  purposes  or  utilizing  combinations  of  media. 

Format  for  Providing  User  Direction 

User  direction  is  provided  by  written  text  referenced  to  a  matrix,  worksheets,  and 
lists  of  media  and  their  features. 

Procedures  and  Criteria 

The  aid  directs  users  to: 


.  * 
■r. 
v. 


■i 

a 


1.  Write  a  behavior,  condition,  and  standard  for  each  learning  objective.  No 
guidance  is  provided  for  this  procedure. 

2.  Analyze  each  objective  to  determine  training  parameters  that  are  relevant  in 
presenting  the  instruction,  forming  an  instructional  strategy,  and  eliciting  student 
responses.  Lists  of  potential  parameters  provided  by  the  aid  include:  (a)  presentation 
parameters  including  visual,  auditory,  tactile,  and  kinesthetic  cues;  (b)  instructional 
strategy  parameters  including  repetition,  knowledge  of  results,  linear,  branching,  and 
adaptive;  and  (c)  student  response  parameters  including  selection,  specific,  created, 
indication,  manipulation,  and  gross  body  movement.  A  user  worksheet  for  recording 
relevant  parameters  for  each  learning  objective  is  mentioned,  but  not  provided. 


3.  Review  a  "Media  Capabilities  Matrix,"  which  rates  the  suitability  of  each  of  29 
generic  media  types  for  providing  the  listed  parameters.  On  the  basis  of  these  suitability 
ratings,  users  are  to  develop  a  set  of  generic  media  candidates  for  each  objective.  The 
users  are  then  directed  to  review  each  candidate  media  and  select  desired  features. 

4.  A  "Generic  Media  Section"  in  the  text  where  each  media  type  and  its  various 
available  features  are  described.  Users  enter  a  set  of  "Data  Grouping  Sheets"  and  use  a 
"Data  Locator  Card"  to  determine  whether  any  specific  medium  can  satisfy  all  of  the 
required  features.  If  more  than  one  medium  satisfies  the  desired  features,  a  specific 
medium  is  selected  based  on  costs.  (The  "Data  Grouping  Sheets"  provide  cost  data  for  the 
year  the  description  of  the  media  selection  aid  was  generated.) 


Outcome  of  Selecting  Process 

At  the  conclusion  of  the  selection  process,  each  user  has  selected  the  least  expensive 
medium  that  satisfies  the  training  requirement. 
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Braby  et  al.  (1975) 


Intended  Users 


This  media  selection  decision  aid  is  designed  for  use  by  instructional  designers  having 
expert  knowledge  of  media. 

Student  Population 

The  target  population  for  the  instruction  consists  of  milivary  students. 

Media  Covered 

Two  sets  of  media  are  provided.  The  first  media  set  is  categorized  by  type  of 
learning.  Each  of  12  types  of  learning  are  assigned  from  2  to  8  primary  media  delivery 
systems  and  from  1  to  5  alternative  media  delivery  systems.  Primary  media  systems  fully 
support  learning  while  alternative  media  systems  only  provide  partial  support.  Each 
media  delivery  system  consists  of  a  single  medium  or  a  media  mix. 

The  second  media  set  consists  of  89  specific  media.  Both  sets  of  media  range  from 
conventional  media  to  computers  and  simulators. 

Format  for  Providing  User  Direction 

User  direction  is  provided  by  a  formal  three-step  selection  procedure  referenced  to 
matrices  and  supporting  texts  and  tables. 

Procedures  and  Criteria 

The  aid  directs  users  to: 

1.  Classify  and  group  training  objectives  according  to  the  type  of  learning  they 
represent.  A  table  gives  the  characteristics  of  training  objectives  and  specifies  what  type 
of  learning  is  associated  with  those  characteristics.  The  users  determine  the  character¬ 
istics  that  are  relevant  to  their  training  objectives  and  identify  the  associated  type  of 
learning. 

2.  Identify  two  or  more  types  of  media  delivery  systems  that  will  support  each 
training  objective.  To  do  this,  users  are  referred  to  the  appropriate  type  of  learning  table 
where  a  matrix  lists  special  criteria  that  may  be  considered  in  selecting  media  and  the 
media  delivery  systems  that  support  those  criteria.  Special  criteria  may  include  any  of  a 
variety  of  stimulus,  training,  and  administrative  criteria  unique  to  specific  training 
programs.  Stimulus  criteria  include  stimulus  characteristics  such  as  video  and  audio  cues, 
and  trainee  movement  requirements.  Training  criteria  include  group  size  and  location. 
Administrative  criteria  include  cost  and  course  development  characteristics.  Users  are 
required  to  check  the  special  criteria  that  are  relevant  to  the  training  objectives.  The 
media  delivery  systems  that  satisfy  the  greatest  number  of  special  criteria  are  selected  as 
candidate  systems. 

If  users  wish  to  consider  special  criteria  or  media  delivery  systems  not  listed  in 
the  table,  an  alternative  matrix  may  be  constructed  using  the  second  set  of  89  media. 
This  procedure  is  only  recommended  for  very  experienced  instructional  media  users. 
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The  aid  also  directs  users  to  compare  the  candidate  systems  in  a  test  of 
practicality  where  the  following  1 1  criteria  are  to  be  considered:  (a)  marginal  technical 
solution,  (b)  state-of-the-art,  (c)  size  of  system,  (d)  interface  with  existing  program,  (e) 
time  needed  or  available  to  produce  system,  (f)  budget  cycle  constraints,  (g)  adoption  of 
innovations,  (h)  courseware  development,  (i)  high  cost  alternatives,  (j)  learning  style  of 
trainees,  and  (k)  other  constraints  (such  as  command  policy  and  existing  investment  in 
production  facilities).  The  aid  instructs  the  users  to  eliminate  any  candidate  systems  that 
do  not  meet  these  criteria,  but  provides  no  guidance  for  rating  the  candidate  systems 
against  the  criteria. 

3.  Estimate  implementation  costs  for  each  of  the  remaining  candidate  systems. 
Users  are  referred  to  a  provided  worksheet  with  detailed  cost  requirements  and  to  record 
cost  requirements  for  each  remaining  candidate  system.  (A  Fortran  IV  program  for 
computing  these  costs  is  available.) 

Outcome  of  Selection  Process 

At  the  conclusion  of  the  selection  process,  each  user  has  selected  the  most  cost- 
effective  instructional  delivery  system  or  mix  of  systems  for  each  training  objective. 


Bretz  (1971) 


Intended  Users 


This  media  selection  decision  aid  is  designed  for  instructional  designers  to  use 
specifically  with  self-contained  communication  systems  having  reproducible  software. 
(Supplementary  media  such  as  audio  and  visual  aids,  which  only  assist  an  instructor  in 
presenting  a  lesson,  are  not  considered.) 

Student  Population 

The  target  population  for  the  instruction  appears  to  be  any  student  audience. 

Media  Covered 


The  range  of  media  is  composed  of  eight  classes  of  media  (audio-motion-visual, 
audio-still-visual,  audio-semimotion,  motion-visual,  still-visual,  semimotion,  audio,  and 
print).  Each  class  consists  of  from  one  to  six  media  for  a  total  of  31  specific  media. 
Specific  media  vary  from  television  and  radio  to  sound-on-slides,  filmstrips,  and  audio 
tapes.  Users  may  combine  classes  of  media  to  create  other  media  classes. 

Format  for  Providing  User  Direction 

User  direction  is  provided  by  text  procedure  referenced  to  a  master  flowchart  and 
several  supplementary  flowcharts. 

Procedures  and  Criteria 


Media  selection  decisions  are  made  in  the  following  manner. 

1.  The  aid  requires  users  to  determine  whether  communication  media  are 
appropriate  by  asking,  "Is  the  presentation  of  sensory  information,  or  the  sending  of  verbal 
messages,  involved?"  If  the  answer  is  yes,  the  users  are  referred  to  the  master  flowchart 
to  determine  whether  the  objective  is  concrete  (best  expressed  in  audio  and/or  visual 
terms)  or  abstract  (best  expressed  in  language).  The  aid  provides  a  supplementary 
flowchart  to  assist  users  in  making  this  decision.  The  objective  is  considered  "concrete- 
visual"  if  there  is  a  requirement  for:  (a)  visual  recognition,  (b)  recognition  or  recall  of  a 
procedure,  (c)  the  understanding  of  a  two-dimensional  relationship,  or  (d)  a  three- 
dimensional  structure.  The  objective  is  considered  "concrete-audio"  if  there  is  a 
requirement  for:  (a)  specific  sounds  or  (b)  an  appreciation  of  music  or  oral  literature. 
Otherwise,  the  objective  is  considered  abstract  and  best  expressed  in  language. 

2.  To  discriminate  further  between  concrete  objectives  that  are  either  audio  or 
visual,  the  users  are  referred  to  another  supplementary  flowchart  where  they  must 
consider  the  requirements  for:  (a)  the  degree  of  motion  (from  none  to  fast),  (b)  whether 
or  not  the  objective  involves  the  recognition  or  performance  of  a  procedure,  and  (c)  if  the 
type  of  movement  of  a  subject  is  an  important  characteristic.  Resolving  these  issues 
leads  the  users  to  one  of  four  possible  media  classes:  audio-motion-visual,  audio-still- 
visual,  motion-visual,  or  still-visual. 

3.  If  the  objective  is  abstract,  the  users  are  referred  to  another  supplementary 
flowchart  to  determine  the  means  of  conveying  narration.  Narration  decision  criteria  are: 
(a)  reading  ability,  (b)  the  need  for  the  personal  element  (speaker's  expression,  hand 
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gestures,  etc.),  feelings,  or  attitude,  (c)  group  versus  individual,  and  (d)  if  oral  or  written 
language  arts  are  being  taught.  The  users'  responses  to  these  decision  criteria,  in  turn, 
lead  them  to  one  of  two  additional  flowcharts:  (a)  print  narration  and  (b)  audio  or  audio 
and  print  narration. 

4.  If  the  outcome  is  print  narration,  several  classes  of  media  are  possible.  To 
determine  which  of  these  media  classes  would  be  appropriate,  the  users  answer  the 
following  questions:  (a)  Are  abstract  concepts  and  relationships  involved  which  could  be 
expressed  graphically?  (b)  Is  recall  of  a  set  of  items  required?  (c)  Is  user  overview  of  the 
outline  of  a  discussion  useful?  (d)  Is  there  any  logical  reason  for  using  full  motion,  buildup 
(presentation  of  symbols  in  relation  to  concrete  subjects),  or  pointing  (coordinating  visual 
presentations  with  audio  narration)?  The  users'  responses  to  these  questions  lead  to 
selection  of  one  of  four  possible  media  classes  (print,  motion-visual,  semimotion,  or  still- 
visual). 

If,  on  the  other  hand,  the  outcome  is  audio  or  audio  with  print  narration,  the 
users  must  still  answer  these  questions  but  also  indicate  if  familiar  spoken  words  are 
related  to  unfamiliar  written  or  printed  ones.  In  this  case,  the  users'  responses  lead  to 
selection  of  one  of  four  different  media  classes  (audio,  audio-motion-visual,  audio¬ 
semimotion,  or  audio-still-visual). 

5.  Once  a  media  class  for  each  objective  has  been  selected,  the  users  are  directed 
to  review  the  selected  classes  and  determine  if  the  number  of  media  classes  can  be 
consolidated  to  reduce  the  variety  of  required  equipment.  Examples  of  how  classes  can  be 
consolidated  are  provided. 

The  aid  also  suggests  considering  practicality  factors  such  as  costs,  resources,  or 
constraints.  No  guidance  is  provided  for  performing  this  procedure. 

Outcome  of  Selection  Process 

At  the  conclusion  of  the  selection  process,  each  user  has  selected  a  class  of  media  for 
presenting  instruction. 
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Collins  et  al.  (1987)  (Training  Decisions  System) 

Intended  Users 

The  Training  Decisions  System  (TDS)  situation/level  decision  aid  is  designed  for  use 
by  manpower,  personnel,  and  training  communities  personnel.  It  is  an  automated  decision 
process  designed  to  assist  these  personnel  in  determining  what,  where,  and  when  to  train 
first-term  airman  skills  while  also  ensuring  effective  training  at  minimum  cost. 

Student  Population 

The  target  population  for  the  instruction  is  Air  Force  enlisted  personnel. 

Format  for  Providing  User  Direction 

The  aid  provides  user  direction  by  a  menu-driven  computerized  system. 

Procedures  and  Criteria 

Data  Base  Development 

Before  TDS  can  be  used  as  a  decision  aid,  data  bases  must  be  established  for  each  of 
the  following  subsystems. 

Task  Characteristics  Subsystem  (TCS).  The  TCS  data  base  is  composed  of  multiple 
elements.  First,  a  preliminary  list  of  all  task  training  objectives  must  be  entered  into  the 
system.  The  TDS  can  use  the  key  task  objective  words  to  group  the  entered  tasks  into 
task  training  modules  (TTMs).  These  groupings  can  also  be  established  or  revised  by 
subject  matter  experts.  This  manual  grouping  capability  allows  parts  of  tasks  to  be 
grouped  separately  for  training  at  different  times  and  in  different  environments. 

Second,  training  sites  for  each  TTM  are  identified  and  added  to  the  data  base  with 
resource  information  for  each  site.  Resource  information  includes  the:  (1)  types  and 
numbers  of  equipment  needed  for  the  instruction  and  (2)  the  student-to-instructor  ratio 
for  the  various  training  locations. 

Third,  the  following  TTM  time  estimates  are  developed  for  the  data  base:  (1)  the 
amount  of  time  currently  spent  providing  instruction,  (2)  the  maximum  amount  of  time 
desired  to  provide  the  instruction,  and  (3)  the  minimum  amount  of  time  needed  for 
instruction  under  a  contingency  situation. 

Field  Utilization  Subsystem  (FUS).  The  current  training  path  for  each  military 
occupational  specialty  (MOS)  is  entered  into  the  FUS  data  base. 

Resource/Cost  Subsystem  (RCS).  The  following  types  of  data  are  entered  into  the 
RCS  data  base:  (1)  the  annual  recurring  costs  for  equipment,  (2)  the  number  of  instructors 
needed  for  the  equipment,  and  (3)  the  student/instructor  ratio. 

User  Procedures 

For  each  training  module,  users  must  specify  potential  training  locations  and  paths. 
For  each  training  path,  the  TDS  indicates  training  costs,  times,  and  equipment  and 
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provides  graphic  displays  for  each  training  path.  Users  can  select  the  least  expensive 
training  path  that  meets  their  needs. 

Outcome  of  TDS  Selection  Process 

selection  process,  each  user  has  selected  a  specific  training 


Gagne  &  Briggs  (1974) 

Intended  Users 

This  media  selection  decision  aid  is  designed  for  use  by  instructional  designers. 

Student  Population 

The  target  population  for  the  instruction  is  any  student  audience. 

Media  Covered 

The  media  choices  are  determined  totally  by  the  users. 

Format  for  Providing  User  Direction 

User  direction  is  provided  by  a  sequence  of  eight  steps  referenced  to  text  material, 
tables,  and  a  worksheet. 

Procedures  and  Criteria 


Users  record  responses  to  each  of  the  following  eight  media  decision  steps  on  a 
provided  worksheet.  The  aid  directs  users  to: 

1.  Identify  the  lesson  objectives  and  refers  them  to  several  text  chapters  to  assist 
in  this  process. 

2.  Classify  the  lesson  objectives  in  terms  of  the  domain  of  learning  outcome  (mode 
of  learning)  and,  if  required,  the  subdomain  (type  of  learning).  The  text  includes  a  chapter 
for  classifying  objectives  but  the  users  are  not  referred  to  it. 

3.  List  the  instructional  events  to  be  accomplished  during  the  lesson.  The  aid 
refers  them  to  a  text  chapter  to  assist  in  this  process. 

4.  Select  the  types  of  stimuli  required  for  each  objective  and  refers  them  to  Dale's 
"cone  of  experience,"  which  is  briefly  discussed  in  the  text.  Users  are  further  instructed 
to  consider  the  nature  of  the  objective,  the  nature  of  the  learner,  and  the  particular 
instructional  event  being  planned.  No  guidance  is  provided  for  selecting  stimulus  types  on 
these  three  criteria. 

5.  List,  for  each  instructional  event,  the  candidate  media  which  can  convey  the 
required  types  of  stimuli  (sound,  color,  motion,  etc.).  The  users  totally  determine  the 
range  of  media.  The  users  are  referenced  to  a  few  examples  of  types  of  stimuli  and 
appropriate  candidate  media.  The  authors  instruct  the  users  to  identify  a  range  of 
candidate  media  and  suggest  that  later  the  user  may  want  to  substitute  other  media  or  to 
reduce  or  enlarge  the  range  of  media,  but  they  are  unclear  as  to  why  or  how  to  do  this. 

6.  Select  the  "theoretically"  best  media  even  if  it  is  beyond  the  available  resources 
(time  or  costs).  No  guidance  is  provided  for  determining  the  best  media. 


7.  Make  a  final  media  selection  and  to  determine  the  practicality  of  using  the 
theoretically  best  media.  Users  are  instructed  to  consider  15  listed  practical  factors. 
These  practical  factors  involve  considerations  of  affective  impact,  adaptability  to 


conditions  of  learning  and  instructional  events,  availability,  cost  effectiveness,  disruption, 
group  size,  programmability,  range  of  viewing  and  hearing  sequence  flexibility,  stimulus 
characteristic  requirements,  storage  requirements,  and  teacher  training  requirements. 
Users  have  the  option  of  adding  additional  factors  to  this  list.  The  aid  directs  us*rs  to 
consider  the  nature  of  the  learners  and  lesson  objectives.  In  making  practical  factor 
decisions,  users  are  directed  to:  (a)  stay  within  the  resources  and  constraints  of  the  media 
developers  and  the  classroom  environment,  (b)  change  media  infrequently  enough  to 
promote  efficiency  but  frequently  enough  to  promote  interest  and  effectiveness,  and  (c) 
use  the  capabilities  of  the  media. 

Outcome  of  Selection  Process 

At  the  conclusion  of  the  selection  process,  the  user  has  selected  a  medium  or  media 
mix  for  each  instructional  event  of  the  lesson  objective. 


Goodman  (1971) 


Intended  Users 

No  specific  users  are  identified. 

Student  Population 

The  target  population  for  the  instruction  appears  to  be  any  student  audience. 
Media  Covered 


The  media  choices  are  determined  totally  by  the  user. 

Format  for  Providing  User  Direction 

User  direction  is  provided  by  a  sequence  of  nine  steps. 

Procedures  and  Criteria 

The  aid  directs  users  to: 

1.  Identify  and  list  anything  the  student  is  required  to  view,  hear,  or  manipulate  in 
performing  the  task. 

2.  List  the  media  or  media  combinations  that  are  not  useful  based  on  student 
characteristics  (age,  grade  level,  IQ,  reading  ability,  physical  characteristics,  cultural 
background,  attitudes,  etc.).  No  guidance  is  provided  for  developing  an  initial  list  or 
media  ror  for  relating  media  characteristics  to  student  characteristics. 

3.  Rank  the  medium  or  media  combinations  according  to  their  hypothesized 
effectiveness  in  providing  for  student  viewing,  hearing,  or  manipulating  and  in  relating  to 
student  characteristics.  No  guidance  is  provided  for  making  these  rankings. 

4.  Determine  the  most  useful  methods  of  presenting  each  medium  or  media 
combination.  To  do  this,  the  aid  instructs  users  to  consider  the  most  effective  "modes" 
(large  group,  small  group,  individual  study,  laboratory)  in  which  the  instruction  could  be 
conducted.  Then,  the  users  are  to  list  specific  methods  for  use  with  each  medium  or 
media  combination.  No  guidance  is  provided  for  linking  a  method  to  a  medium  or  media 
combination. 

5.  Determine  if  the  media  identified  in  step  4  are  available  or  affordable  and 
eliminate  any  media  that  do  not  meet  these  criteria. 

6.  Determine,  for  each  of  the  media  or  media  combinations,  what  presently 
available  instructional  materials  would  help  the  students  accomplish  specified  behavioral 
objectives  and  what  additional  instructional  materials  would  be  useful.  No  guidance  is 
provided  for  making  these  determinations. 

7.  Analyze  the  production  capabilities  and  finances  that  are  locally  available  for 
the  production  of  additional  useful  instructional  materials  that  are  not  commercially 
available.  From  this  analysis,  users  are  supposed  to  determine  which  of  the  needed 
instructional  materials  could  be  locally  produced. 
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8.  Rank  order  the  remaining  medium  or  media  combinations  in  terms  of  their 
combined  instructional  and  cost  effectiveness.  No  guidance  is  provided. 

9.  Select  the  most  cost-effective  medium  or  media  combination,  again  without 
guidance. 

Outcome  of  Selecting  Process 

At  the  conclusion  of  the  selection  process,  the  user  has  selected  the  most  cost 
effective  medium  or  media  combination  for  presenting  instruction. 
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Holden  (1974) 


Intended  Users 


This  media  selection  decision  aid  is  designed  for  use  by  instructional  designers. 
Student  Population 

The  target  population  for  the  instruction  is  any  student  audience. 

Media  Covered 


Seven  types  of  "software"  are  considered:  printed  text,  audio  tape,  slides  or 
filmstrip,  synchronized  tape  with  slide  or  filmstrip,  16mm  motion  picture,  videotape,  and 
computer  program.  The  decision  aid  also  includes  a  separate  listing  of  hardware  required 
to  support  the  software. 

Format  for  Providing  User  Direction 

User  direction  is  provided  by  a  sequence  of  steps  referenced  to  text  material, 
matrices,  and  an  example. 

Procedures  and  Criteria 


1.  For  each  learning  objective,  users  are  directed  to  select  a  restriction  rating  for 
each  of  seven  decision  factors:  (a)  degree  of  control,  (b)  kind  of  learned  capability,  (c) 
cost,  (d)  flexibility  of  use,  (e)  population  restrictions,  (f)  adaptability,  and  (g)  compati¬ 
bility  with  hardware.  To  aid  in  assigning  ratings,  the  text  defines  each  decision  factor  and 
assigns  it  ratings  that  reflect  the  "levels  of  restriction"  that  each  factor  might  have 
relative  to  the  learning  objective.  For  example,  the  decision  factor  "population,"  has  been 
assigned  the  following  three  ratings:  (a)  no  restrictions,  (b)  language  restrictions  (either 
language  spoken  or  intellectual  level),  and  (c)  logistic  restrictions. 

2.  For  each  type  of  software,  users  are  directed  to  delete  the  decision  factors  that 
do  not  meet  the  rating  code  (restriction  criteria)  established  for  decision  factors  in  step  I. 
Users  are  referred  to  a  "software  selection"  matrix  that  specifies  restriction  ratings  for 
each  of  the  seven  types  of  software  with  respect  to  the  seven  decision  factors. 

3.  The  aid  directs  users  to  select  those  software  types  that  have  no  decision  factors 
deleted  or,  if  all  have  some  deleted,  those  that  have  the  fewest. 

4.  If  more  than  one  type  of  software  satisfies  all  decision  factors,  users  are 
instructed  to  look  for  combinations  of  these  remaining  types  of  media  software  that  might 
provide  variety  and  optimize  training  effectiveness.  A  partial  example  of  this  process  is 
provided. 


5.  If  all  of  the  software  types  have  some  decision  factors  deleted,  users  are 
directed  to  review  the  responses  and  decide  if  any  of  them  can  be  changed.  No  guidance 
is  provided  for  deciding  which  responses  can  be  changed  or  how  they  should  be  changed. 
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6.  If  several  potential  types  of  software  remain,  users  are  instructed  to  opt  for  cost 
savings  first  and  variety  second.  Ratings  of  cost  restrictions  are  provided  in  the  matrix. 

7.  The  aid  directs  users  to  select  hardware  using  the  same  procedures  as  performed 
for  software  selection.  The  user  is  directed  to  a  hardware  selection  matrix.  This  matrix 
lists  5  decision  factors  (cost,  compatibility  between  models,  portability,  life  of  equipment, 
and  maintenance  cost)  that  intersect  with  11  types  of  hardware  (4  types  of  projectors,  3 
types  of  tape  players,  2  types  of  video,  1  type  of  optical  reader,  and  1  type  of  computer). 
Additionally,  the  aid  rates  each  decision  factor  (low,  medium,  or  high)  to  indicate  how 
appropriate  it  is  relative  to  each  type  of  hardware. 

8.  After  inappropriate  types  of  hardware  have  been  eliminated,  users  are  directed 
to  consider  the  remaining  hardware  types  in  terms  of  their  match  with  the  different  forms 
of  software  and  the  "universality"  of  their  use.  No  guidance  is  provided  for  making  these 
considerations.  The  aid  also  warns  users  not  to  use  the  acquisition  cost  of  the  hardware  as 
a  deciding  factor  without  considering  hardware  maintenance  costs  and  period  of  use. 

9.  Finally,  the  aid  directs  users  to  select  a  media  system  based  on  the  software  and 
hardware  selections.  At  this  point,  if  more  than  one  choice  exists,  the  aid  indicates  that 
the  final  decision  may  be  made  subjectively. 

Outcome  of  Selection  Process 


At  the  conclusion  of  the  selection  process,  the  user  has  selected  a  single  medium  or 
media  combination  for  presenting  instruction  and  specified  both  the  software  and 
supporting  hardware  components. 
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Kemp  (1980) 


Intended  Users 


No  specific  users  are  identified. 

Student  Population 

The  target  population  for  the  instruction  appears  to  be  any  student  aut'ience. 
Media  Covered 


The  range  of  media  is  composed  of  eight  categories  (photographic  print  series,  slide 
series,  filmstrips,  recordings,  overhead  transparencies,  motion  pictures,  videotape  re¬ 
cordings,  and  multi-image/multimedia).  The  media  choices  within  each  category  are 
determined  by  the  users. 

Format  for  Providing  User  Direction 

User  direction  is  provided  by  a  sequence  of  three  questions  and  a  written  prescrip¬ 
tion,  all  of  which  are  referenced  to  flowcharts  and  tables. 

Procedures  and  Criteria 


The  aid  directs  users  to: 

1.  Answer  three  questions:  (a)  Which  teaching/learning  pattern  (presentation, 
individualized  learning,  or  small  group  interaction)  is  most  appropriate,  considering  the 
objective  and  the  nature  of  the  student  group,  (b)  which  category  of  learning  experiences 
(direct  realistic  experiences,  verbal  or  printed  word  abstractions,  or  vicarious  sensory 
experience)  is  most  suitable  for  the  objective  and  instructional  activity  in  terms  of  the 
selected  teaching/learning  pattern,  and  (c),  if  sensory  experience  is  indicated  or  selected, 
which  attributes  of  media  are  necessary  or  desirable.  Limited  textual  guidance  is 
provided  in  the  instructions  for  answering  these  questions. 

2.  Users  are  then  referred  to  an  appropriate  media  selection  flowcharts  for  the 
selected  teaching/learning  pattern.  Each  chart  has  up  to  three  "levels."  At  each  level, 
questions  are  asked  about  media  attributes  with  answers  to  be  based  on  responses  to  the 
initial  three  questions.  Level  one  questions  concern  the  type  of  experience  required  for 
the  learner  (real,  vicarious  or  sensory,  or  verbal  abstraction).  Level  two  questions  address 
the  need  for  audio  and/or  visual  stimulus,  and  level  three  questions  consider  whether  still 
or  motion  displays  are  needed.  Media  choices  are  indicated  based  on  responses  to  these 
questions. 

3.  When  media  choices  lead  to  a  group  of  related  media,  such  as  motion  pictures, 
the  aid  directs  users  to  choose  the  most  practical  form.  To  assist  in  this  decision,  they 
are  referred  to  a  table  which  lists  nine  empirical  factors  to  use  in  comparing  the  various 
types  of  media.  These  include  commercial  availability,  preparation  costs,  reproduction 
costs,  time  to  repair,  skills  and  services  required,  viewing  and  handling,  maintenance  and 
storage,  student  preference,  and  instructor  preference. 

4.  The  aid  directs  users  to  consider  relative  planning,  production,  and  duplication 
costs.  Users  are  referred  to  a  table  providing  examples  of  how  such  costs  can  be 
documented  and  compared. 


r 


$ 

ft 

§ 

OT 

W 

ss 


jAjft^AjftVtfAM^tjtfA* A*.fa1.li*Al1 AlA*.lttJt11  M  ij-  h*  if  4 


1  It*  |(*  JJ*Ja  a 


■■■■■ 


IJB 


1 


M 


•<5 

$ 

ft: 

•»' 


I 


Outcome  of  Selection  Process 

At  the  conclusion  of  the  selection  process,  the  user  has  selected  a  single  medium  or 
media  mix  for  presenting  instruction. 
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Kribs  et  al.  (1983)  (Automated  Instructional  Media  Selection) 

Intended  Users 

The  Automated  Instructional  Media  Selection  (AIMS)  decision  aid  is  designed  for  use 
by  instructional  designers.  Media  specialists  working  together  with  educational  specialists 
and  subject  matter  experts  could  make  optimum  use  of  this  decision  aid. 

Student  Population 

The  target  population  for  the  instruction  appears  to  be  any  student  audience. 

Media  Covered 

The  media  choices  are  determined  totally  by  the  users.  Up  to  99  different  media  can 
be  compared  at  once.  The  AIMS  system  provides  a  list  of  22  specific  media  for  user 
consideration.  These  media  vary  from  conventional  media  to  simulators,  and  computer- 
assisted  or  computer-managed  instruction. 

Format  for  Providing  User  Direction 

User  direction  is  provided  by  a  series  of  steps  referenced  to  worksheets  for  rating 
media.  These  ratings  are  summarized  in  a  computer  data  base.  A  "User  Guide"  is 
provided  for  entering  information  into  the  menu-driven  computer  data  base  program. 

Procedures  and  Criteria 

The  aid  directs  users  to: 

1.  Develop  a  list  of  media  or  combinations  of  potential  media.  They  have  the 
option  of  using  a  media  list  provided  by  AIMS.  The  final  list  of  selected  media  is  put  into 
the  AIMS  media  data  base. 

2.  List  all  the  instructional  characteristics  the  media  should  address  (such  as  the 
modes  of  learning,  types  of  learning,  stimulus  characteristics,  etc.).  Again,  users  have  the 
option  of  using  an  instructional  characteristics  list  provided  by  AIMS.  The  final  list  of 
instructional  characteristics  is  also  put  into  the  AIMS  data  base.  From  these  data  lists, 
the  computer  creates  and  prints  a  media-by-attribute  worksheet.  No  guidance  is  provided 
for  determining  the  completeness  or  appropriateness  of  either  the  media  list  or  the 
instructional  characteristics  list. 

3.  Subjectively  rate  each  medium  or  media  combination  (0-5)  on  its  ability  to 
deliver  each  instructional  characteristic.  These  ratings  are  put  into  the  computer.  The 
AIMS  system  then  sums  the  ratings  and  rank  orders  the  media  in  accordance  with  their 
ratings.  No  guidance  is  provided  for  determining  the  rating  values.  In  summing  the 
ratings,  all  instructional  characteristics  are  treated  as  being  equally  important. 

Outcome  of  Selection  Process 

At  the  conclusion  of  the  selection  process,  the  users  have  generated  a  list  of  media 
and/or  media  combinations  ranked  according  to  their  perceived  appropriateness  for 
presenting  instruction  for  the  given  objective. 
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Leiblum  (1980) 


Intended  Users 


This  media  selection  decision  aid  is  designed  for  use  by  instructional  technologists. 
The  decision  aid  is  specifically  concerned  with  whether  computer-assisted  learning  (CAL) 
is  the  most  suitable  medium  for  the  solution  of  the  instructional  problem  under 
consideration. 

Student  Population 

The  target  population  for  the  instruction  appears  to  be  any  student  audience. 

Media  Covered 

The  media  choices  are  determined  totally  by  the  user. 

Format  for  Providing  User  Direction 


User  direction  is  provided  by  a  sequence  of  steps  used  in  conjunction  with  a  matrix 
worksheet  and  a  list  of  learning  delivery  mechanisms. 

Procedures  and  Criteria 

The  aid  directs  users  to  build  a  media  selection  matrix  for  each  task  or  instructional 
event,  taking  into  account  any  local  limitations  or  requirements,  in  the  following  manner. 

1.  The  aid  directs  users  to  enter  available  or  easily  accessible  media  that  are  to  be 
considered  as  the  column  headings  on  a  worksheet. 

2.  The  aid  directs  users  to  examine  six  learning  delivery  mechanisms  listed  as  row 
headings  on  the  worksheet;  (a)  instructional  event,  (b)  learner,  (c)  sensory  modality 
(stimulus  characteristics),  (d)  production,  (e)  distribution,  and  (f)  other.  Each  delivery 
mechanism  contains  from  four  to  eight  specific  attributes.  Brief  explanations  of  each 
learning  delivery  mechanism  attribute  are  provided.  For  each  medium  to  be  considered, 
users  are  directed  to  indicate  whether  each  attribute  is  (a)  obtainable,  (c)  obtainable,  but 
with  difficulty,  or  (c)  unobtainable. 

3.  The  aid  directs  users  to  study  each  instructional  task  or  event  and  identify  and 
list  (on  a  separate  piece  of  paper)  all  behavioral  objectives  and/or  critical  behaviors.  To 
do  this,  the  users  are  instructed  to  obtain  required  documentation. 

k.  The  aid  directs  users  to  assign  a  weighting  factor  (WF)  for  each  attribute 
according  to  its  relative  importance  in  achieving  the  critical  behavior  and  to  enter  it  on 
the  worksheet. 

5.  The  aid  directs  users  to  multiply  the  attribute  WF  by  the  obtainability  rating  of 
the  medium  to  get  an  overall  score. 

6.  The  aid  directs  users  to  determine  whether  the  medium  with  the  highest  score 
can  provide  all  of  the  essential  attributes.  If  it  cannot,  an  alternative  medium  or  media 
mix  must  be  selected.  To  determine  alternative  media,  the  users  are  directed  to  identify 
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those  media  with  the  highest  totals  and  select  a  medium  or  media  mix  that  provides  all 
the  essential  attributes. 

Outcome  of  Selecting  Process 

At  the  conclusion  of  the  selection  process,  the  user  has  selected  a  single  medium  or  a 
media  mix  for  presenting  instruction. 
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Levie  (1975) 


Intended  Users 


This  media  selection  decision  aid  is  designed  for  use  by  instructional  developers. 
Student  Population 

The  target  population  for  the  instruction  appears  to  be  any  student  audience. 
Media  Covered 

The  media  choices  are  determined  totally  by  the  user. 

Format  for  Providing  User  Direction 


User  direction  is  provided  by  a  four-step  flowchart  referenced  to  text  instructions. 
Procedures  and  Criteria 

In  selecting  task  objectives,  the  aid  suggests  that  users  deal  with  small  tasks  and  be 
exact  about  the  class  of  instructional  event  or  type  of  learning  objective  involved.  Users 
make  media  selection  decisions  in  the  following  manner. 

1.  The  aid  directs  users  to  determine  the  required  media  attributes  for  each 
objective.  The  aid  describes  five  media  attribute  categories:  sign  vehicle  characteristics 
(e.g.,  words  vs.  pictures),  realism  cue  characteristics  (e.g.,  color  vs.  black  and  white, 
motion  vs.  still),  sensory  channel  characteristics  (e.g.,  seeing  vs.  hearing  vs.  seeing  plus 
hearing),  loci  of  control  characteristics  (e.g.,  fixed  vs.  flexible  pace  or  sequence),  and 
response  acceptance  characteristics  (e.g.,  overt  vs.  convert  responding).  No  guidance  is 
provided  for  determining  which  of  these  attributes  apply  to  the  learning  objective. 

2.  To  determine  media  candidates,  users  are  directed  to  analyze  what  the  aid  refers 
to  as  the  "non-media  components"  of  the  instructional  situation.  Non-media  components 
include:  (a)  learning  task  analysis,  (b)  learner  analysis,  and  (c)  an  analysis  of  the  learning 
environment.  (The  aid  assumes  that  non-media  decisions  have  previously  been  deter¬ 
mined.)  Examples  of  how  to  base  media  decisions  on  non-media  components  are  provided. 
For  example,  the  aid  states  that  an  auditory  medium  should  not  be  used  for  deaf  learners. 
No  further  guidance  for  identifying  media  candidates  from  non-media  components  is 
provided. 

3.  The  aid  directs  users  to  apply  pragmatic  constraints.  The  aid  states  that  there 
are  two  "real  world"  constraints:  production  constraints,  which  are  associated  with 
creating  the  first  copy  of  the  instructional  product  and  dissemination  constraints,  which 
are  associated  with  making  and  delivering  multiple  copies  to  the  places  where  they  will  be 
used.  The  aid  lists  and  briefly  discusses  six  production  and  six  dissemination  constraints 
to  guide  the  users  in  assessing  the  practicality  of  each  medium  on  the  alternative  media 
list.  No  further  guidance  for  applying  pragmatic  constraints  is  provided. 

h.  To  make  a  final  choice,  the  aid  directs  users  to  choose  the  medium  that  (a)  most 
closely  approximates  the  conditions  in  which  the  learner  will  be  expected  to  perform,  (b) 
allows  the  learner  to  respond  in  the  same  manner  as  in  the  criterion  situation,  and  (c) 
allows  the  learner  to  make  the  largest  number  of  relevant  responses  in  a  given  period. 
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The  aid  cautions  users  against  automatically  selecting  the  latest  "in"  medium.  No  further 
guidance  is  provided  for  the  final  selection  of  media. 

Outcome  of  Selection  Process 

At  the  conclusion  of  the  selection  process,  the  user  has  selected  a  single  medium  for 
presenting  the  instruction. 
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Lonigro  &  Eschenbrenner  (1973) 


Intended  Users 


This  media  selection  decision  aid  is  designed  for  use  by  media  specialists  and  training 
managers  of  technical/vocational  education. 

Student  Population 

The  target  population  for  the  instruction  consists  of  technical/vocational  students. 
Media  Covered 


Eight  media  types  are  considered:  still  pictures/graphics,  motion  pictures,  television, 
simulation,  audio  recordings,  programmed  instruction,  tape/slide,  and  computer-assisted 
instruction. 

Format  for  Providing  User  Direction 

User  direction  is  provided  by  a  sequence  of  five  steps  referenced  to  text  material,  a 
matrix,  worksheet,  and  examples. 

Procedures  and  Criteria 


The  aid  directs  users  to: 

1.  Determine  which  of  five  types  of  learning  comprise  the  job  task:  (a)  learning 
factual  information,  (b)  learning  multiple  discriminations,  (c)  learning  principles,  con¬ 
cepts,  and  rules,  (d)  learning  procedures,  and  (e)  learning  to  perform  skilled  perceptual 
motor  acts. 

2.  Specify  the  proficiency  level  required  for  each  task  (high,  partial,  or  low). 
Proficiency  definitions  are  provided  but  no  guidance  is  provided  for  determining  pro¬ 
ficiency  requirements. 

3.  Develop  a  list  of  media  that  would  ensure  achievement  of  the  instructional 
objective.  To  develop  this  list,  the  aid  provides  users  with  a  matrix  that  crosses  eight 
types  of  media  with  the  five  types  of  learning.  Each  type  of  media  is  rated  according  to 
the  proficiency  it  provides  for  each  type  of  learning  (high  partial,  or  low).  Users  are 
instructed  to  list  the  media  types  from  the  matrix  that  meet  or  exceed  the  proficiency 
requirements  for  the  type  of  learning  that  the  task  involves. 

4.  Specify  an  acceptable  level  of  production  costs  that  includes  costs  for  equip¬ 
ment,  supplies,  facilities,  and  personnel.  A  table  lists  relative  cost  ranges  of  the  eight 
media  in  five  levels  from  low  to  high.  Users  may  add  additional  cost  production 
considerations  to  the  table. 

5.  Select  the  most  cost-proficient  medium.  Users  are  referred  to  a  "gameboard" 
worksheet  on  which  to  plot  the  position  of  potential  media  in  terms  of  costs  and 
proficiency.  Users  then  select  the  medium  that  provides  the  highest  proficiency  at  the 
lowest  cost.  The  aid  indicates  that  selected  media  should  provide  at  least  a  partial 
proficiency  rating  at  no  more  than  moderate  costs.  Media  choices  that  provide  high 
proficiency  at  high  costs  require  further  study  before  selection.  No  basis  is  provided  for 


The  Marine  Corps  Systems  Approach  to  Training  User's  Guide  (Draft)  (19S7) 

Intended  Users 

Although  the  media  selection  decision  aid  was  designed  by  the  Marine  Corps,  no 
specific  Marine  Corps  users  are  identified. 

Student  Population 

The  target  population  for  the  instruction  appears  to  consist  of  students  enrolled  in 
Marine  Corps  Development  and  Education  Command  formal  schools. 

Media  Covered 

A  total  of  15  specific  media  are  considered.  These  media  vary  from  simple  charts 
and  posters,  conventional  films,  and  terrain  model  boards  to  the  more  sophisticated  media 
technology  such  as  simulators,  computer-assisted  instruction,  and  videodiscs. 

Format  for  Providing  Direction 

User  direction  is  provided  by  a  written  selection  procedure  with  references  to  a 
series  of  tables. 

Procedures  and  Criteria 

The  aid  directs  users  to: 

1.  Categorize  terminal  and  enabling  learning  objectives  according  to  the  general 
kind  of  learning  (mode  of  learning)  they  represent  (affective,  cognitive,  or  motor). 
However,  in  order  to  use  the  tables  provided,  users  must  categorize  training  objectives 
according  to  the  specific  type  of  earning  behaviors  (type  of  learning)  that  are  involved 
(for  example,  remembering  facts,  making  decisions,  or  manipulating  equipment  controls). 

2.  Identify  sets  of  potential  media  based  on  (a)  the  learning  behaviors  and  (b)  the 
training  content.  However,  no  guidelines  for  choosing  media  in  terms  of  training  content 
are  provided. 

3.  Review  lists  of  training  methods  and  media  characteristics.  The  apparent 
purpose  of  this  review  is  to  determine  which  media  are  appropriate  for  the  training 
methods  to  be  used  and  which  media  characteristics  are  desirable.  Listed  methods  include 
action  images,  brainstorming,  buzz  groups,  case  studies,  computer-assisted  instruction, 
demonstrations,  discussions,  embedded  training  games,  in-basket  exercises,  job  aids, 
lectures,  modeling,  peer  mediated  learning,  practical  application,  programmed  instruc¬ 
tion,  related  readings,  roleplay,  and  simulation.  However,  no  basis  is  provided  for 
selecting  training  methods  or  for  relating  media  characteristics  to  training  methods  or  to 
any  of  the  previously  identified  characteristics  of  the  training  situation. 

4.  Make  a  tentative  media  selection  decision  based  on  the  kind  of  learning  and  the 
type  of  training  content.  No  guidance  is  provided  for  choosing  media  based  on  training 
content. 

5.  Modify  the  selected  media  set  based  on  specified  physical  and  academic 
characteristics  of  the  target  population  and  the  available  resources  (personnel,  equipment, 
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facilities,  and  time).  No  direction  is  provided  for  modifying  the  selected  media  sets  based 
on  these  criteria. 

In  addition  to  these  selection  criteria,  coded  lists  of  functional  attributes  of 
media  and  media  characteristics  are  provided  that  can  be  used  to  identify  desirable  media 
and  methods  of  presentation.  However,  no  guidelines  are  provided  for  using  these  lists  or 
for  integrating  functional  attributes  with  other  criterion  in  selecting  media. 

6.  Make  a  final  selection  based  on  experience  or  preferences. 

Outcome  of  Selection  Process 

At  the  conclusion  of  the  selection  process,  the  user  has  selected  several  possible 
media  from  which  a  final  choice  of  media  can  be  made. 


Media  Selection  (1974) 


Intended  Users 


No  specific  users  are  identified. 

Student  Population 

The  target  population  of  the  instruction  appears  to  >e  any  student  audience. 
Media  Covered 

The  media  choices  are  determined  totally  by  the  user. 

Format  for  Providing  User  Direction 


User  direction  is  provided  by  a  six-step  text  procedure  referenced  to  examples. 
Procedures  and  Criteria 

The  decision  aid  requires  users  to  identify  lesson  learning  objectives  before  using  the 
aid.  No  guidance  is  provided  for  this  procedure.  The  aid  directs  users  to: 

1.  Identify  the  potential  media  for  presenting  the  instructional  materia.  Users  are 
advised  to  consider  three  guidelines: 

a.  Special  signal  needs  (content  stimulus  characteristics)  such  as  audio  or 
visual  attributes. 

b.  Media  with  costs  within  the  range  of  available  funds.  The  aid  reminds  users 
to  include  expens  s  ~ ttached  to  production,  operation,  and  maintenance  costs.  At  this 
point,  several  potential  media  are  to  be  identified  and  their  relative  costs  compared. 


ments). 


:.  The  changeability  of  content  areas  (frequency  of  modification  require- 


At  the  end  of  step  1,  users  are  expected  to  have  developed  a  list  of  media  they 
are  willing  to  consider  based  on  the  three  guidelines.  However,  no  basis  is  provided  for 
determining  which  types  of  media  might  satisfy  the  guideline  requirements. 

2.  Eliminate  any  media  that  cannot  be  justified  in  terms  of  cost  and  time 
expenditures,  availability  of  professional  and  technical  assistance,  functioning  staffs,  and 
faculty.  Again  no  guidance  is  provided  for  eliminating  media  on  the  basis  of  these 
considerations.  By  the  end  of  step  2,  users  have  narrowed  their  list  of  media  to  those  they 
are  willing  to  consider. 

3.  Eliminate  or  combine  media  on  the  basis  of  basic  instructional  needs.  Users  are 
instructed  to  (a)  identify  the  basic  instructional  needs  to  be  satisfied,  (b)  determine  how 
well  each  prospective  medium  meets  those  needs,  and  (c)  eliminate  any  media  that  cannot 
meet  those  needs.  To  determine  whether  selected  media  satisfy  basic  instructional  needs, 
a  worksheet  is  provided  that  identifies  basic  or  essential  instructional  capabilities.  These 
include  introductory  capabilities  (objectives,  motivation,  etc.)  expository  capabilities 
(generalities,  supporting  explanations,  etc.)  inquisitory  capabilities  (practice,  test,  etc.) 
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and  flexibility  and  adaptiveness  capabilities  (identification  of  learner  needs  and  dynamic 
modifications  and  helps).  No  further  guidance  is  provided  for  performing  this  step. 
However,  examples  of  media  are  provided  before  and  after  being  combined.  Users  are 
instructed  to  rate  the  media  in  terms  of  introductory,  expository,  flexibility,  and 
adaptiveness  capabilities. 

4.  Construct  a  media  selection  algorithm  and  apply  it  to  all  objectives.  An  example 
of  a  media  selection  algorithm  is  presented  as  a  flowchart  of  questions  with  yes/no 
responses.  In  the  example,  factors  addressed  include  the  level  of  learning  (familiarization 
or  instructional)  mode  of  learning,  stimulus  characteristics,  type  of  learning,  simulation 
requirements,  and  interaction  requirements.  Responses  to  the  questions  lead  to  the 
selection  of  one  or  more  of  the  listed  media  options.  No  guidance  for  developing  the 
algorithm  is  provided. 

5.  Consolidate  media  within  lessons  where  different  media  have  been  selected  for 
different  objectives  and  it  is  practical  to  reduce  the  types  of  media  required.  The  stated 
purpose  is  to  prevent  over-mediation  of  the  students.  No  guidance  for  consolidating 
media  is  provided. 

6.  Reexamine  selections  for  the  entire  hierarchy  and  revise  where  necessary.  No 
guidance  is  provided. 

Outcome  of  Selection  Process 

At  the  conclusion  of  the  selection  process,  the  user  has  selected  a  medium  for 
presenting  each  instructional  objective. 


Merrill  &  Goodman  (1972) 


Intended  Users 

This  media  selection  decision  aid  is  designed  for  use  by  public  school  teachers, 
administrators,  and  curriculum  specialists. 

Student  Population 

The  target  population  for  the  instruction  appears  to  consist  of  public  school  students. 
Media  Covered 


The  range  of  media  is  composed  of  six  categories:  objects,  still  pictures,  motion 
pictures,  audio  material,  written  material,  and  combinations  of  media.  Each  category 
consists  of  from  three  to  seven  media.  A  total  of  22  specific  media  are  considered. 
Specific  media  are  limited  to  conventional  delivery  systems  such  as  filmstrips,  models, 
and  commercial  16mm  films. 

Format  for  Providing  User  Direction 

User  direction  is  provided  by  a  systematic  five-step,  partially  proceduralized 
selection  guide  which  references  tables  and  written  guidelines. 

Procedures  and  Guidelines 

To  use  the  guide,  terminal  and  enabling  objectives  must  be  identified  and  training 
strategy  prescriptions  planned  for  each  objective.  Media  selection  decisions  are  made  in 
the  following  manner. 

1.  Users  write  a  statement  of  the  instructional  outcome  including  a  description  of 
the  audience,  the  behavior  to  be  performed,  the  conditions  under  which  the  behavior  will 
be  observed,  and  the  degree  of  acceptable  performance  (the  elements  of  a  performance 
objective).  A  description  and  example  of  each  of  these  elements  are  provided. 

2.  Users  determine  the  appropriate  domain  (mode  of  learning)  for  each  objective. 
Written  descriptions  and  examples  of  each  domain  are  also  provided. 

3.  For  each  objective,  users  determine  the  specific  instructional/learning  strategy 
(type  of  learning)  that  is  appropriate  within  the  specified  domain.  Written  guidelines  and 
lists  indicate  the  instructional/learning  strategies  that  may  occur  under  each  domain. 
Definitions  and  examples  of  each  instructional/learning  strategy  are  also  provided. 

<f.  For  each  objective,  users  write  a  strategy  prescription  for  each  training  function 
(instructional  event);  that  is,  how  the  objective  is  to  be  presented,  practiced,  and 
evaluated.  The  strategy  prescription  indicates,  for  each  training  function,  what  kinds  of 
information  will  be  provided  to  the  student  and  what  kinds  of  responses  the  student  will  be 
required  to  make.  Again,  written  guidelines  and  examples  as  well  as  printed  worksheets 
to  assist  in  entering  data  are  provided.  However,  users  must  determine  which  strategies 
are  appropriate  for  each  type  of  training  content. 

5.  Users  are  required  to  write  separate  media  prescriptions  for  each  training 
function  (evaluation,  practice,  and  presentation)  taking  into  account  the  objective, 


instructional/learning  strategy,  and  any  special  student  needs.  No  guidance  is  given  for 
identifying  student  needs. 

Users  are  directed  to  a  separate  set  of  media  tables  for  each  training  function. 
They  are  instructed  to  select  a  media  category  for  each  of  the  three  training  functions  by 
reading  a  description  of  each  category  and  determining  which  is  the  most  appropriate  for 
the  instructional/learning  strategy  involved. 

Once  a  media  category  has  been  selected,  the  aid  refers  users  to  charts  that  list 
media  in  the  identified  media  category  and  their  key  attributes.  Users  select  the  medium 
or  media  combination  that  best  meets  the  requirement  of  the  media  prescription. 
Examples  of  the  selection  process  and  worksheets  are  provided. 

Outcome  of  Selection  Process 

At  the  conclusion  of  the  media  selection  process,  the  user  has  selected  a  medium  or 
media  combination  for  presenting  instruction  for  each  training  function  of  the  objective. 


Methods/Media  Selection  Guidelines  (1977) 


Intended  Users 

The  Naval  Air  Maintenance  Training  Group  (NAMTRAGRU)  decision  aid  is  designed 
for  use  by  NAMTRAGRU  course  development  and  training  peisonnel. 

Student  Population 

The  target  population  for  the  instruction  consists  of  NAMTRAGRU  students. 

Media  Covered 


The  range  of  media  is  composed  of  nine  categories.  Each  category  consists  of  from 
one  to  four  specific  media.  Specific  media  vary  from  conventional  media  to  trainers  and 
simulators. 

Format  for  Providing  User  Direction 

User  direction  is  provided  by  a  sequence  of  steps  referenced  to  tables  and/or  lists. 
Procedures  and  Criteria 


Although  not  stated,  the  procedure  assumes  that  users  have  developed  a  list  of 
training  objectives.  The  decision  aid  is  separated  into  two  parts:  selecting  an 
instructional  method  and  selecting  a  medium.  The  aid  directs  users  to: 

1.  Select  an  instructional  method  for  each  objective. 

a.  For  each  objective,  users  are  directed  to  choose  one  or  more  instructional 
methods  for  each  of  three  stages  of  learning  (stages  of  training)  using  a  stage  of  learning 
by  instructional  method  matrix. 

b.  Users  are  referred  to  an  appropriate  instructional  method  table  that  defines 
the  method  and  provides  appropriate  conditions  for  its  use,  qualifications  for  its 
applications,  and  guidelines  for  using  the  method.  The  conditions  may  indicate  the  stage 
of  learning,  level  of  performance,  type  of  learning,  mode  of  learning,  function  of  media, 
level  of  student  ability,  etc.  for  which  the  method  is  recommended.  Not  all  conditions 
appear  in  every  instructional  method  table.  Users  compare  the  recommended  methods 
and  select  one. 

2.  Select  a  medium  or  media  mix. 

a.  Users  are  directed  to  classify  and  group  the  learning  objectives  into  one  of 
eight  listed  types  of  learning. 

b.  Users  are  directed  to  select  two  or  more  candidate  media  for  each  type  of 
learning  by  using  a  media  selection  decision  tree  and  a  matrix  that  compares  the 
effectiveness  of  instructional  media  for  different  types  of  learning.  Users  are  guided 
through  the  media  selection  decision  tree  by  yes/no  responses  that  consider  the  following 
criteria:  content  stimulus  characteristics  (audio,  visual,  and  written),  function  of  media 
(practice  and  simulation),  and  simulation  fidelity.  This  process  identifies  one  or  more 
appropriate  categories  of  media  for  the  obj.'tive.  Users  then  use  the  matrix  that 


compares  the  effectiveness  of  instructional  media  for  different  types  of  learning  to 
determine  the  relative  effectiveness  of  the  selected  media  category  (low,  average,  or 
high).  No  guidance  is  provided  for  relating  the  method  and  media  selection  processes. 


c.  Users  are  directed  to  estimate  the  cost  of  using  each  candidate  category  of 
media  to  train  the  required  number  of  students  to  meet  the  objectives.  A  table  gives 
approximate  costs,  advantages/disadvantages,  and  uses  for  specific  types  of  media. 

d.  Users  are  directed  to  identify  the  most  cost-effective  medium  that  meets 
both  the  instructional  method  and  type  of  learning  requirements.  A  table  specifies  (1) 
specific  types  of  media  for  each  category,  (2)  the  approximate  hardware  and/or  software 
costs,  (3)  production  time,  and  (4)  whether  or  not  the  software  can  be  locally  produced. 
However,  no  guidance  is  provided  for  performing  this  step. 

Outcome  of  Selection  Process 


At  the  conclusion  of  the  selection  process,  the  user  has  selected  either  a  single 
medium  or  a  media  mix  for  presenting  instruction. 


Nonpersonnel  Studies  and  Analysis  Services  for  Assessment 
of  New  Training  Technologies  (1985) 

Intended  Users 

This  media  selection  decision  aid  is  designed  for  the  Air  Force  to  use  for  replacing 
traditional  instruction  with  advanced  technology  training  delivery  (ATTD)  systems. 

Student  Population 

The  target  population  for  the  instruction  consists  of  Air  Force  students. 

Media  Covered 

Six  specific  media  are  considered:  computer-based  training,  intelligent  computer- 
assisted  instruction,  interactive  videodisc,  maintenance  simulators,  embedded  training, 
and  advanced  job  performance  aids. 

Format  for  Providing  User  Direction 


User  direction  is  provided  by  a  combination  of  text  procedures,  charts,  worksheets, 
and  formulas. 

Procedures  and  Criteria 

User  direction  procedures  employ  the  Multi-Attribute  Utility  Analysis  (MAUA) 
(Keeney  &  Raiffa  1976).  Procedures  are  described  in  terms  of  an  example.  The  steps  in 
the  example  are  as  follows. 

1.  The  overall  number  of  students  per  year,  number  of  students  per  class,  and 
number  of  hours  of  instruction  per  student  must  be  specified  for  the  user.  Also,  general 
training  situation/level  information  (such  as  level  of  student  education,  whether  the 
training  is  preliminary  or  advanced,  etc.)  must  be  determined  for  or  by  the  user  since  this 
information  is  required  in  the  decision  process. 

2.  Potential  media  that  are  not  appropriate  for  the  training  situation/level  or  are 
not  sufficiently  ’’mature"  to  be  used  at  this  time  are  rejected.  No  guidance  is  provided  for 
these  decisions.  Student  learning  objectives  are  also  to  be  provided  to  the  user  at  this 
time. 

3.  The  aid  directs  the  user  to  determine  training  costs.  The  Instructional  Systems 
Development  (ISD)  process  is  examined  to  determine  the  relative  costs  and  effectiveness 
of  each  of  the  remaining  media.  (In  practice,  media  only  vary  in  cost  for  the  development 
and  implementation  steps  of  the  ISD  process;  so  only  those  two  steps  are  considered.) 

Costs  for  the  development  of  each  of  the  media  are  determined  from  a  table  of 
estimated  costs.  Implementation  costs  are  determined  by  the  users.  A  provided  listing  of 
types  of  potential  implementation  costs  consists  of  staffing/training,  facilities  require¬ 
ments,  hardware,  software,  and  the  scheduling/updating  of  courseware.  For  each  type  of 
cost,  a  number  of  considerations  are  listed. 

Many  costs  would  not  be  expected  to  vary  with  the  type  of  medium.  In  practice, 
only  those  costs  that  would  be  expected  to  vary  are  considered.  Also,  the  costs  of 
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equipment  or  materials  already  available  are  not  counted.  For  example,  if  computer 
terminals  are  already  available,  no  acquisition  cost  for  computer  terminals  would  be 
included.  After  all  development  and  implementation  costs  have  been  estimated,  they  are 
totaled  to  obtain  the  final  cost  for  each  medium. 

4.  The  aid  directs  the  user  to  determine  training  effectiveness  by  listing  presenta¬ 
tion  "features"  that  are  relevant  to  the  training  content  and  then  rating  each  potential 
medium  on  its  strength  for  each  feature.  Any  number  of  relevant  features  might  be 
considered.  The  example  includes  features  such  as  degree  of  interaction,  individualiza¬ 
tion,  and  ease  of  updating  materials.  However,  it  is  not  completely  clear  how  to 
determine  relevant  features.  Users  are  referred  to  a  table  of  10  factors  that  might  be 
considered  in  selecting  media.  These  factors  include  increased  control,  reduced  resource 
requirements,  individualization,  timeliness  and  availability,  reduced  training  time,  im¬ 
proved  job  performance,  convenience,  change  agent,  increased  learning  satisfaction,  and 
reduced  development  time.  However,  at  least  some  of  the  features  listed  in  the  example 
do  not  appear  in  the  table  of  factors.  The  derivation  of  those  features  is  unclear.  Also, 
the  aid  provides  no  guidance  for  determining  which  factors  are  relevant  to  a  particular 
instructional  setting  or  content. 

In  rating  potential  media  for  effectiveness,  a  total  of  100  points  is  assigned  to 
each  feature.  Users  divide  the  100  points  among  the  prospective  media  based  on  how 
effective  they  perceive  each  medium  to  be  for  that  feature.  For  example,  if  two  media 
are  being  considered,  a  user  might  assign  50  points  to  each,  60  to  one  and  40  to  the  other, 
etc.  Users  also  weight  each  feature  in  terms  of  its  relative  importance.  Since  the  sum  of 
all  the  weights  must  add  up  to  1.00,  each  feature  is  assigned  some  decimal  portion  of  that 
value  (.2,  .05,  etc.). 

Next,  feature  scores  are  computed  for  each  medium  by  multiplying  the  feature's 
weight  by  the  assigned  rating  for  that  medium.  For  example,  suppose  computer-based 
instruction  is  assigned  a  rating  of  50  for  the  feature  "individualization"  and  individualiza¬ 
tion  is  assigned  a  weight  of  .12.  The  score  for  computer-based  instruction  on  the 
individualization  feature  is  then  50  x  .12  =  6.  Finally,  a  total  score  is  computed  for  each 
medium  by  summing  its  feature  scores. 

5.  The  aid  directs  users  to  identify  the  most  cost-effective  medium.  After  cost  and 
effectiveness  scores  are  totaled  for  each  medium,  they  are  plotted  on  a  graph  with 
effectiveness  on  the  vertical  scale  and  costs  on  the  horizontal  scale.  A  medium  high 
score  in  effectiveness  and  a  low  score  in  cost  are  considered  to  be  desirable.  However, 
how  to  select  a  medium  if  one  is  lower  in  cost  but  another  is  higher  in  effectiveness  is  not 
explained. 

In  addition  io  the  procedures  described  above,  this  aid  provides  a  number  of 
tables  that  appear  to  be  relevant  to  media  selection  but  are  not  referenced  in  the 
procedures.  One  table  indicates  which  media  are  appropriate  for  specified  "instructional 
strategies"  (drill  and  practice,  simulation,  problem  analysis,  etc.)  and  specifies  related 
developmental  costs.  Another  table  lists  strengths  and  weaknesses  of  media  in  terms  of 
certain  "characteristics."  These  characteristics  appear  to  relate  to  the  features  that  are 
considered  in  developing  effectiveness  ratings.  Another  table  lists  strengths  for  each  of 
the  considered  media  and  gives  some  general  types  of  training  situations  or  content  for 
which  they  would  be  appropriate.  Another  table  lists  instruction,  management,  and 
testing  applications  which  should  be  considered  in  selecting  media.  No  indication  is  given 
as  to  how  these  tables  should  be  employed  in  selecting  media. 
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Intended  Users 


Pieper  et  al.  (1978)  (Training  Developer  Decision  Aid) 


The  Training  Developer  Decision  Aid  (TDDA)  is  designed  for  use  by  training 
development  specialists  (TDS). 

Student  Population 

The  target  population  for  the  instruction  consists  of  Army  enlisted  personnel. 

Media  Covered 


The  14  specific  media  considered  vary  from  conventional  media  to  programmed  text, 
real  equipment,  and  simulators. 

Format  for  Providing  User  Direction 

User  direction  is  provided  by  a  text  description  of  the  development  and  use  of  the 
TDDA  process. 

Procedures  and  Criteria 


Although  not  stated,  the  TDDA  assumes  that  the  user  has  developed  a  list  of  training 
objectives.  The  TDDA  process  is  composed  of  three  "partitions":  (1)  training  prescrip¬ 
tions,  (2)  training  hierarchies  and  sequences,  and  (3)  training  costs.  Users  make  selection 
decisions  in  the  following  manner. 

1.  In  partition  one,  a  training  prescription  is  developed  for  each  task.  A  training 
prescription  consists  of  a  (a)  training  algorithm,  (b)  stimulus  medium,  (c)  response 
acceptance  mechanism,  (d)  training  method,  and  (e)  training  setting.  The  aid  directs  the 
users  to: 


a.  Select  one  of  12  listed  training  algorithms  to  meet  the  task  objective.  The 
aid  states  that  the  verb  used  to  describe  the  task  should  be  used  in  selecting  an 
appropriate  training  algorithm. 

b.  Select  one  of  five  classes  of  stimuli  (verbal,  audio,  visual,  audio-visual,  or 
tactile)  based  on  yes/no  answers  to  15  "stimulus  questions." 

c.  Select  one  or  more  of  four  classes  of  responses  (equipment  manipulation, 
voice,  written,  or  body  movement)  based  on  yes/no  responses  to  14  response  acceptance 
questions. 


d.  Select  a  compatible  stimulus  media  and  response  acceptance  mechanism. 
Users  are  provided  with  lists  of  15  training  methods,  10  stimulus  media  and  14  response 
acceptance  method  mechanisms.  To  determine  the  characteristics  common  to  the  method 
and  stimulus  media,  users  assess  them  in  terms  of  the  pacing  controller,  stimuli  content, 
and  the  next  learning  activity  (sequence  controller).  To  determine  the  characteristics 
common  to  the  method  and  the  response  acceptance  mechanisms,  users  assess  them  in 
terms  of  the  pacing  controller,  type  of  evaluation,  feedback,  and  next  learning  activity. 
The  purpose  of  this  procedure  is  to  select  a  method  that  best  satisfies  the  three 
characteristics  of  the  stimulus  medium  and  the  four  characteristics  of  the  response 
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acceptance  mechanism.  However,  it  is  not  clear  how  these  decisions  are  to  be  made. 
Users  are  also  instructed  to  ensure  that  the  task  verb  of  the  algorithm  is  appropriate  for 
the  method.  The  author  notes  that  a  list  of  the  methods  that  are  compatible  with  each 
algorithm  has  been  developed.  However,  the  list  is  neither  in  the  text  nor  referenced. 

e.  Determine  the  training  setting.  Users  are  provided  with  a  flowchart 
requiring  yes/no  responses  to  questions  about  the  number  of  students,  the  nature  of 
student  interactions  (individual/team),  and  whether  or  not  the  task  is  equipment  related. 
The  outcome  of  this  flowciart  indicates  one  of  five  possible  training  settings  (small  team 
site,  large  team  site,  individual  carrel,  small  group  carrel,  or  traditional  classroom). 

2.  In  partition  two,  training  hierarchies  and  sequences  are  established.  The  aid 
directs  users  to: 

a.  Identify  equipment  and  supporting  equipment  (test  equipment,  etc.)  for  each 
task  and  each  pay  grade  level.  (This  assumes  the  task  involves  equipment  maintenance  or 
operation.) 

b.  Assign  each  task  a  training  priority.  To  assign  priorities,  users  must  rate 
the  task  in  terms  of  (1)  how  long  after  initial  training  the  soldier  will  be  expected  to 
perform  the  task,  (2)  its  contribution  to  the  systems's  mission,  (3)  a  combination  of  time 
to  application  and  task  learning  time,  (4)  the  percentage  of  members  performing  the  task, 
and  (5)  the  percentage  of  time  spent  performing  the  task. 

c.  Assign  tasks  to  training  periods.  To  do  this,  users  must  determine  (1)  total 
training  time  available  for  each  training  option  and  (2)  required  training  times  for 
individual  tasks.  The  highest  priority  tasks  are  assigned  to  resident  formal  school  training 
until  the  total  amount  of  resident  training  time  is  used  up.  The  next  highest  priority  tasks 
are  assigned  to  on-the-job  training  (03T)  until  OJT  training  time  is  used  up.  Any 
remaining  tasks  are  not  trained. 

3.  In  partition  three,  costing  procedures  are  used  to  determine  training  costs  for 
resident  training  and  COT.  To  determine  these  costs,  users  are  directed  to  consider 
numerous  variables  such  as  the  number  of  hours  available  in  a  learning  cycle,  the  number 
of  hours  required  to  learn  the  tasks,  and  the  percentage  of  nonproductive  time  (used  for 
unit  movement,  deployment,  or  other  nontraining  activities,  etc.).  The  training  method  is 
also  used  as  a  factor  in  determining  costs.  Cost  rankings  are  provided  for  each  training 
method  (previously  identified  in  partition  one). 

Outcome  of  Selection  Process 

At  the  conclusion  of  the  selection  process,  the  user  has  selected  a  training  situation, 
media,  and  method  for  presenting  instruction  and  determined  the  relative  costs. 
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Reiser  &  Gagne  (1983) 


Intended  Users 

This  media  selection  decision  aid  is  designed  for  use  by  instructional  designers. 

Student  Population 

The  target  population  for  the  inst-uction  seems  to  be  any  student  audience. 

Media  Covered 

The  20  specific  media  considered  vary  from  conventional  media  to  actual  equipment, 
simulators,  training  devices,  and  interactive  TV. 

Format  for  Providing  User  Direction 

User  direction  is  provided  by  a  flowchart. 

Procedures  and  Criteria 


Users  make  media  selection  decisions  in  the  following  manner. 

1.  Although  not  stated  in  the  media  selection  decision  aid  itself,  prior  to  using  the 
decision  aid,  each  learning  objective  must  be  classified  into  one  of  five  domains  of 
learning  outcomes  (modes  of  learning)  and  the  instructional  setting  (location  and  group 
size)  must  be  determined.  No  guidance  is  given  for  determining  any  of  these  prerequisite 
data.  However,  the  aid's  authors  indicate  that  different  domains  of  learning  outcomes 
require  certain  types  of  feedback  capabilities  and  that  some  media  are  more  appropriate 
for  providing  these  capabilities  than  are  others. 

2.  At  the  first  decision  point  in  the  flowchart,  users  are  required  to  determine 
whether  or  not  an  error  in  the  actual  performance  of  the  task  has  serious  consequences. 
If  the  consequences  are  serious,  the  users  must  select  a  medium  for  practicing  the  task 
from  a  list  of  media  options.  Users  are  then  directed  to  proceed  to  the  second  decision 
point  (whether  or  not  a  practice  medium  is  required). 

3.  At  the  next  decision  point,  users  are  required  to  determine  whether  or  not  the 
training  should  be  (a)  centrally  broadcast,  (b)  self-instructed  (with  readers  or  nonreaders), 
or  (c)  led  by  an  instructor  (with  readers  or  nonreaders).  For  each  of  these  conditions, 
users  are  directed  to  sets  of  potential  media  based  on  the  domain  of  learning  outcome 
(attitude,  verbal  information,  mental  skill,  or  motor  skill)  and  the  need  for  visuals.  For 
self-instruction  with  nonreaders,  feedback  requirements  are  also  considered. 

Outcome  of  Selection  Process 


At  the  conclusion  of  the  selection  process,  the  user  has  selected  a  specific  medium,  a 
CnOiuc  uf  fn Cum,  Oi  a  mix  of  media  for  presenting  instruction. 
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Romiszowski  (1974) 


Intended  Users 


This  media  selection  decision  aid  is  designed  for  use  by  teachers  and  lesson  designers. 
Student  Population 

The  target  population  for  the  instruction  is  any  student  audience. 

Media  Covered 

The  16  general  types  of  media  considered  vary  from  live  commentary  and  written 
text  to  special  training  devices  and  simulators. 

Format  for  Providing  User  Direction 


User  direction  is  provided  by  a  sequence  of  steps  referenced  to  flowcharts  and  text 
descriptions. 

Procedures  and  Criteria 

1.  The  aid  directs  users  to  define  the  behavioral  objectives  for  each  topic.  No 
procedures  are  provided  for  performing  this  step. 

2.  The  aid  directs  users  to  prescribe  the  instructional  event  for  each  objective.  No 
procedures  are  provided  for  performing  this  step.  However,  the  aid  suggests  that  the 
instructional  events  should  be  expressed  in  terms  of  stimulus  and  response  requirements  so 
that  these  requirements  can  be  matched  later  with  media  attributes. 

3.  The  aid  directs  users  to  assess  (a)  the  type  of  responses  required  of  the  students, 
(b)  the  type  of  stimuli  required  to  communicate  the  message,  and  (c)  the  student  group 
characteristics.  Users  are  provided  with  flowcharts  for  identifying  each  factor.  Each 
flowchart  identifies  one  or  more  media  that  satisfy  the  specified  requirements. 

a.  Type  of  responses  required.  To  identify  the  type  of  responses  required  of 
the  students,  users  must  consider  whether  motor  repsonses  or  perceptual  skills  are 
involved,  how  new  the  task  is  to  the  student,  whether  the  normal  job  situation  would  give 
clear  feedback  on  progress,  and  how  complex  any  verbal  responses  are. 

b.  Type  of  stimuli  required.  To  identify  the  type  of  stimuli  required  to 
communicate  the  message,  users  must  first  consider  visual  aspects  such  as  (1)  whether  or 
not  the  task  deals  with  concrete  observable  objects  or  effects,  (2)  if  important  aspects  of 
the  object  are  hidden  or  obscured,  (3)  if  the  object  is  available,  (4)  the  size  of  the  object, 
(5)  if  three-dimensional  stimuli  are  required,  and  (6)  if  students  must  learn  to  recognize  or 
duplicate  motions  or  actions. 

Users  must  then  identify  verbal  and  sound  stimuli  required  to  communicate 
the  message  by  considering  (1)  whether  or  not  sound  is  an  integral  part  of  the  topic,  (2) 
whether  verbal  communication  is  the  main  objective,  (3)  if  the  topic  is  complex  or 
abstract,  (4)  whether  the  audio  content  can  be  prepared  as  a  script,  and  (5)  whether  the 
topic  is  taught  regularly  without  change. 
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c.  Student  group  characteristics.  To  identify  the  characteristics  of  the  student 
group,  users  must  first  decide  if  they  control  the  class  size.  If  not,  media  based  on  group 
size  and  individualization  of  study  are  prescribed.  If  users  control  group  size,  they  must 
consider  if  the  training  objective  requires  full  retention  and  is  easily  retained  or  if  it  is  of 
higher  order  (e.g.,  discrimination,  concept,  or  rule)  and  involves  social  skills,  psycho¬ 
motor  skills,  or  perceptual  training. 

i 't.  The  aid  directs  users  to  list  all  the  media  identified  in  step  3  that  satisfy  the 
specified  requirements.  Any  media  that  are  unsuitable  due  :o  environmental  constraints 
are  identified  by  answering  six  yes/no  questions.  These  questions  address  (a)  whether  or 
not  the  media  identified  are  obtainable,  economical  in  regard  to  payoff,  student-proof, 
and  maintainable/repairable;  (b)  if  space  is  available  for  them;  (c)  if  logistics  support 
(power  supplies,  etc.)  are  available;  and  (d)  if  they  will  have  a  reasonable  amount  of  use 
before  subject  matter  changes  render  the  medium  obsolete.  Any  medium  with  a  "no” 
answer  is  deleted.  From  the  revised  list,  users  delete  any  medium  unlikely  to  suit  the 
students  due  to  learning  habits,  past  experiences,  or  psychological  characteristics.  To  do 
this,  users  are  referred  to  the  text  for  a  discussion  of  learning  task  factors  and  student 
factors. 


Next,  users  are  directed  to  assess  whether  or  not  each  medium  satisfies  the  task 
and  student  factor  requirements.  Little  guidance  is  provided  for  this  fairly  subjective 
procedure.  At  the  completion  of  this  step,  the  user's  media  list  should  be  reduced  to  a 
few  candidate  choices. 

5.  The  aid  directs  users  to  select  a  final  medium  or  media  mix.  Users  must 
consider  the  (a)  personal  preferences  of  the  teachers  and  (b)  desirability  of  using  the  same 
media  for  different  training  objectives. 

Outcome  of  Selecting  Process 

At  the  conclusion  of  the  selection  process,  the  user  has  identified  single  medium  or 
media  mix  for  presenting  instruction. 
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User's  Manual  for  Predicting  Military  Task  Retention  (1985) 


Intended  Users 


The  User's  Manual  for  Predicting  Military  Task  Retention  (UMPMTR)  situation/level 
decision  aid  is  designed  for  Army  unit  trainers  to  use  in  determining  how  frequently 
specific  military  occupational  specialty  (MOS)  tasks  require  refresher  training  to  maintain 
a  specified  level  of  proficiency. 

Student  Population 

The  target  population  for  the  decision  aid  consists  of  Army  enlisted  personnel  who 
have  been  previously  trained  to  proficiency  on  a  specified  task. 

Format  for  Providing  User  Direction 

User  direction  is  provided  by  a  sequency  of  three  steps.  User  direction  may  be 
provided  by  a  User's  Manual  or  a  computer-based  User’s  Decision  Aid  (Apple  software  and 
IBM  Basic). 

Procedures  and  Criteria 


The  development  and  use  of  a  UMPMTR  data  base  to  determine  refresher 
requirements  consists  of  two  steps,  the  development  of  a  data  base  and  the  determination 
of  retention  predictors. 

1 .  Data  Base  Development.  The  purpose  of  step  1  is  to  rate  the  retention  difficulty 
of  each  MOS  task.  Step  1  is  performed  by  SMEs  who  are  familiar  with  the  MOS  tasks  for 
which  instruction  is  to  be  developed.  SME  raters  complete  a  10  question  algorithm  to 
assign  each  task  a  retention  score. 

2.  Retention  Prediction  Determination.  Step  2  may  be  performed  by  any  user  and 
does  not  require  SME  inputs.  The  user  is  directed  to  determine  the  no-practice  interval; 
that  is,  time  that  has  elapsed  since  the  task  of  interest  was  last  practiced.  The  user  is 
then  referenced  to  a  performance-prediction  matrix  on  which  task  retention  scores 
intersect  with  length-of-time-without-practice  to  give  a  retention  prediction  value.  This 
value  indicates  the  percentage  of  soldiers  in  the  unit  that  can  perform  the  task  correctly 
after  a  specified  time  period  without  practice.  Given  the  task  retention  score  for  a  task 
and  the  no-practice  interval,  the  user  can  determine  what  percentage  of  the  personnel  are 
likely  to  have  retained  the  task.  Based  on  this  percentage,  users  or  other  decision  makers 
can  determine  whether  retraining  on  the  task  is  required  at  this  time. 

Outcome  of  PMTR  Process 


At  the  conclusion  of  PMTR  process,  the  user  has  determined  the  refresher  training 
intervals  required  to  ensure  specified  levels  of  unit  proficiency  for  each  task. 


Walker  (1965) 


Intended  Users 


This  media  selection  decision  aid  is  designed  for  the  "training  man." 

Student  Population 

The  target  population  for  the  instruction  is  any  student  audience. 

Media  Covered 

A  total  of  16  "training  techniques"  are  considered.  (The  author  apparently  uses  the 
term  "training  technique"  to  cover  a  mixture  of  methods  and  media.)  These  techniques 
vary  from  lectures  and  texts  to  sleep  teaching  and  simulators. 

Format  for  Providing  User  Direction 

User  direction  is  provided  by  written  text  referenced  to  a  matrix  and  an  example. 
Procedures  and  Criteria 

This  decision  aid  assumes  that,  prior  to  the  selection  of  a  training  technique,  a  task 
analysis  has  been  completed  and  other  relevant  information  such  as  instructor  availability, 
class  size,  number  of  trainees,  and  training  location  have  been  established.  Training 
techniques  are  selected  in  the  following  manner. 

1.  Users  are  directed  to  develop,  for  each  objective,  a  list  of  criteria  important  to 
the  selection  of  a  training  technique.  Examples  of  criteria  include  the  type  of  learning, 
updating  requirements,  costs,  preparation  time,  and  availability  of  instructors. 

2.  Users  are  referenced  to  a  "Selection  Criteria  Matrix,"  which  lists  34  selection 
criteria  that  intersect  with  16  training  techniques.  Each  technique  has  been  given  a 
desirable  relationship  rating  (1-5)  for  each  criteria.  The  selection  criteria  are  categorized 
as  either  management-centered  (such  as  time  to  produce,  effectiveness  of  teaching  motor 
skills,  cost  to  teach,  student  intelligence,  etc.)  or  student-centered  (such  as  student 
motivation,  reinforcement,  feedback  retention,  pace,  etc.).  Users  are  directed  to  identify 
the  criteria  in  the  matrix  that  match  those  established  in  step  1. 

3.  Users  are  then  directed  to  sum  the  ratings  of  all  student-centered  criteria  and 
all  other  criteria  they  identified  as  relating  to  the  objective.  Users  are  directed  to  rank 
order  the  desirability  of  each  training  technique  based  on  these  sums.  No  guidance  is 
provided  for  ranking  user  identified  criteria  not  identified  in  the  matrix. 

4.  Users  are  then  directed  to  select  the  technique  with  the  highest  overall  rating 
for  application.  However,  before  selecting  a  training  technique,  users  are  instructed  to 
make  sure  that  its  training  effectiveness  justifies  its  cost.  No  guidance  is  provided  for 
this  procedure. 

Outcome  of  Selection 

At  the  conclusion  of  the  selection  process,  the  user  has  selected  a  single  training 
technique  for  presenting  instruction. 
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CROSS-REFERENCE  TO  DECISION  AID  EXAMPLES 

This  report  provides  recommendations  for  the  development  of  military  training 
decision  aids.  Although  none  of  the  reviewed  training  decision  aids  satisfy  all  of  the 
recommendations  made  in  this  report,  many  of  the  procedures  and  criteria  this  report  has 
advocated  have  been  applied  in  these  aids.  The  following  cross-reference  indicates 
examples  of  existing  training  decision  aids  that  provide  procedures  or  criteria  cited  in 
these  recommendations.  Indicated  aids  should  be  referenced  as  examples  in  developing 
training  decision  aids  for  military  users.  Descriptions  of  all  referenced  training  decision 
aids  are  provided  in  Appendix  A.  Complete  references  for  each  of  these  aids  are  provided 
in  the  reference  section  of  this  report. 


Develop  training  decision  aids  for  specific  military 


jlications. 


The  following  are  examples  of  training  decision  aids  developed  for  specific 
military  applications. 

a.  Aids  for  users  establishing  training  situations/levels. 

(1)  An  aid  for  assisting  military  personnel  in  the  initial  establishment  of 
training  situations/levels  was  developed  by  Pieper,  Guard,  and  Kordek  (1978).  However, 
its  procedures  are  complex  and  should  be  simplified. 

(2)  An  aid  for  assisting  military  personnel  to  compare  the  cost  effective¬ 
ness  of  different  training  sites  was  developed  by  Collins,  Hernandez,  Ruck,  Vaughn, 
Mitchell,  and  Rueter  (1987). 

(3)  An  aid  for  assisting  military  personnel  to  determine  refresher  training 
periods  was  developed  by  the  Army  (User's  Manual  for  Predicting  Task  Retention,  1985). 

b.  Aids  for  users  selecting  media. 

(1)  Aids  for  assisting  military  instructors  to  select  media  were  developed 
by  the  Marine  Corps  (The  Marine  Corps  Systems  Approach  to  Training  User's  Guide 
(Draft),  1987)  and  the  Navy  (Methods/Media  Selection  Guidelines,  1977).  However, 
neither  of  these  aids  concentrates  on  the  needs  of  instructors  and  neither  is  adequate  in 
terms  of  the  format  and  criteria  utilized,  the  clarity  of  the  procedures,  and  the  level  of 
guidance  provided. 

(2)  An  aid  for  assisting  military  course  designers  or  planners  to  select 
media  developed  by  Braby,  Henry,  Parrish,  and  Swope  (1975).  Again,  however,  the  format, 
criteria,  and  procedures  are  inadequate. 

(3)  An  aid  for  assisting  military  personnel  to  replace  conventional  instruc¬ 
tion  with  new  technologies  was  developed  by  the  Air  Force  (Nonpersonnel  Studies  and 
Analysis  Services  for  Assessment  of  New  Training  Technologies,  1985).  Again,  however, 
the  format,  criteria,  and  procedures  are  inadequate. 

2.  Combine  computerized  flowchart  and  matrix  formats  in  structuring  decision 

aids. 

This  report  recommends  combining  flowchart  and  matrix  formats  in  establishing 
training  situations/levels  or  selecting  media,  which  none  of  the  reviewed  training 


V.v'V 


Va'a’a 


decisions  aids  does.  However,  examples  using  flowchart  formats  are  provided  by  Anderson 
(1983),  Bretz  (1971),  Levie  (1975),  Reiser  and  Gagne  (1982),  and  Romiszowski  (1974). 
Examples  using  matrix  formats  are  provided  by  Brc.by  et  al.  (1975),  and  Walker  (1965). 


This  report  also  recommends  computer  automation  of  selection  procedures.  An 
automated  training  decision  aid  for  determining  the  cost  effectiveness  of  alternative 
training  sites  was  developed  by  Collins  et  al.  (1987).  An  automated  aid  for  determining 
refresher  training  was  developed  by  the  Army  (User's  Manual  for  Predicting  Task 
Retention,  1985). 

The  only  automated  media  selection  aid  reviewed  in  this  report  was  developed  by 
Kribs,  Simpson,  and  Mark  (1983).  However,  this  media  selection  system  is  considered 
inadequate  in  terms  of  the  level  of  direction  provided  to  the  user. 

3.  Provide  detailed  guidance  for  training  decision  aids. 

Aids  with  strong  user  guidance  for  establishing  training  situations/levels  were 
developed  by  Pieper  et  al.  (1978)  for  initial  selection  of  training  sites,  Collins  et  al.  (1987) 
for  cost  comparisons  of  training  sites,  and  the  Army  (User's  Manual  for  Predicting  Task 
Retention,  1985)  for  determining  periods  for  refresher  training. 

Examples  of  aids  with  strong  user  guidance  for  selection  of  media  are  provided 
by  Reiser  and  Gagne  (1982)  in  a  flowchart  format,  Wagner  (1965)  in  a  matrix  format, 
Braby  et  al.  (1975)  for  analyzing  costs,  and  the  Air  Force  (Nonpersonnel  Studies  and 
Analysis  Services  for  Assessment  of  New  Training  Technologies,  1985)  for  comparing  the 
cost  effectiveness  of  new  technology. 

4.  Develop  decision  aids  for  specific  types  of  tasks. 

Examples  of  aids  using  the  task  as  the  unit  of  training  are  provided  by  Kribs  et 
al.  (1983),  Leiblum  (1980),  Levie  (1975),  and  Lonigro  and  Eschenbrenner  (1973).  None  of 
the  reviewed  aids  use  functional  areas  or  task  types  as  the  unit  of  training.  However, 
Braby  et  al.  (1975)  provides  specialized  media  selection  models  for  different  types  of 
learning.  Conceivably,  similar  types  of  specialized  media  selection  models  could  be 
constructed  for  different  functional  areas  or  task  types. 

5.  Select  training  situations/levels  based  on  training  priorities,  task  prerequisites, 
training  time,  task  appropriateness,  and  training  costs. 

Training  priorities,  task  prerequisites,  and  training  time  are  discussed  in  Pieper 
et  al.  (1978).  However,  more  efficient  procedures  and  clearer  directions  need  to  be 
developed.  Task  appropriateness  was  not  discussed  as  a  criterion  for  training  settings  or 
levels  by  any  of  the  aids.  Training  costs  for  training  settings  are  dealt  with  by  Collins  et 
al.  (1987). 

6.  Use  existing  situation/level  decision  aids  for  determining  relative  usts  and 
refresher  training  intervals. 

Collins  et  al.  (1987)  is  recommended  for  use  in  determining  relative  costs.  The 
Army's  User's  Manual  for  Predicting  Task  Retention  (1985)  is  recommended  for  use  in 
determining  refresher  training  intervals. 


Select  training  presentation  methods  based  on  training  location,  group  character¬ 


istics.  training  events,  stimulus  characteristics,  affective  (motivational)  requirements. 


rogramming  requirements 


and  requirements  for  automation  of  testing  and  data  management. 


Training  presentation  methods  should  be  established  independently  prior  to  the 
selection  of  training  media.  Training  location  and  group  characteristics  should  be  used 
first  to  reduce  the  number  of  possible  choices. 

Aids  that  contain  procedures  for  selecting  training  methods  separately  from 
training  media  were  developed  by  the  Marine  Corps  (The  Marine  Corps  Systems  Approach 
to  Training  User's  Guide  (Draft),  19S7),  the  Navy  (Methods/Media  Selection  Guidelines, 


1977),  and  the  Army  (Pieper  et  al.,  1978).  However,  none  of  these  aids  provides  a 
procedure  for  basing  media  choices  on  the  selected  training  methods  as  is  recommended  in 
this  report. 

An  example  of  an  aid  that  uses  training  location  and  group  characteristics  as 
criteria  in  media  selection  is  provided  by  Anderson  (1983).  An  example  of  an  aid  that 
varies  media  recommendations  based  on  instructional  events  is  provided  by  Reiser  and 
Gagne  (1982).  An  example  of  an  aid  that  relates  media  selection  to  stimulus  features  is 
provided  by  Kemp  (1980).  Examples  of  aids  that  relate  affective  modes  of  learning  in  the 
form  of  motivation  to  training  media  selection  are  provided  by  Anderson  (1983)  and  Reiser 
and  Gagne  (1983).  Again,  however,  none  of  these  aids  apply  the  specified  criteria  to  the 
selection  of  training  methods  as  is  recommended  in  this  report. 

None  of  the  reviewed  aids  specify  methods  for  dealing  with  retention  require¬ 
ments  or  criteria  for  determining  where  programming  of  instruction  or  automation  is 
required. 

None  of  the  reviewed  aids  provide  criteria  for  determining  when  group  practice 
should  be  utilized.  An  example  of  an  aid  with  criteria  for  use  of  simulations  is  provided 
by  Romiszowski  (1974).  However,  Romiszowski  does  not  provide  criteria  for  determining 
the  type  of  simulation  and  the  degree  of  fidelity  required. 

None  of  the  reviewed  aids  provide  criteria  for  determining  if  automated  testing 
and/or  data  management  methods  are  required. 

8.  Select  training  media  based  on  training  presentation  methods;  requirements  for 
modification,  mobility,  and  development  time:  costs;  availability;  and  personal 


preferences. 

As  previously  stated,  aids  that  provide  procedures  for  selecting  training  presen¬ 
tation  methods  separately  from  training  media  were  developed  by  the  Marine  Corps  (The 
Marine  Corps  Systems  Approach  to  Training  User's  Guide  (Draft),  1987),  and  the  Army 


(Pieper  et  al.,  1978).  However,  none  of  these  aids  provide  procedures  for  basing  media 
choices  on  the  selected  training  methods  as  recommended  in  this  report. 

Practical  requirements  such  as  ease  of  revision,  mobility  of  training  devices,  and 
development  time  are  mentioned  in  several  media  selection  aids.  However,  none  of  the 
reviewed  aids  provide  detailed  guidance  as  to  which  media  systems  best  satisfy  these 
requirements. 


rt 
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Two  types  of  cost  data  have  been  provided  by  the  media  selection  aids  reviewed 
in  this  report,  comparative  and  exact.  Comparative  cost  data  are  useful  for  making  rough 
estimates  concerning  the  relative  expense  of  different  media  (e.g..  high,  medium,  or  low). 
Good  examples  of  comparative  cost  matrices  have  been  provided  by  Lonigro  and 
Eschenbrenner  (1973).  Exact  cost  data  are  typically  difficult  to  procure  and  require 
considerable  knowledge  with  respect  to  the  characteristics  of  the  training  facilities, 
maintenance  requirements,  travel  requirements,  etc.  A  good  example  of  an  exact  cost 
model  for  media  selection  is  provided  by  Braby  et  al.  (1975). 

The  availability  of  media  and  the  personnel  preference  of  the  user  are  obvious 
criteria  that  need  no  examples  or  discussion. 

9.  Consider  a  combination  of  training  presentation  methods  and  media  for  teaching 
each  task. 

None  of  the  aids  considered  requirements  for  mixes  of  training  methods.  Aids 
that  consider  requirements  for  mixes  of  media  were  developed  by  Braby  et  al.  (1975), 
Gagne  and  Briggs  (1974),  Goodman  (1971),  Holden  (1974),  Kemp  (1980),  Kribs  et  al.  (1983), 
Leiblum  (1980),  Merrill  and  Goodman  (1972),  the  Navy  (Methods/Media  Selection  Guide¬ 
lines,  1977),  Reiser  and  Gagne  (1983)  and  Romiszowski  (1974).  ~~ 


I 

I 

I 


1 

I 


DISTRIBUTION  LIST 


Headquarters  Marine  Corps  (Code  TAP-31)  (5) 

Commanding  Officer,  Naval  Training  Systems  Center 
Naval  Training  Systems  Center  (Technical  Library)  (5) 

Education  Center,  Quantico,  VA 

Commander,  U.S.  AR1,  Behavioral  and  Social  Sciences  (PERI-POT-I) 

Chief,  U.S.  AR1  USAREUR  (Library)  (2) 

Commander,  Air  Force  Human  Resources  Laboratory,  Brooks  Air  Fcrce  Base,  TX 
Director,  Training  Systems  Development,  Randolph  Air  Force  Base,  TX  (HQ  ATAC/XPRS) 
Director  of  Research,  U.S.  Naval  Academy 
Center  for  Naval  Analyses 

Defense  Technical  Information  Center  (DTIC)  (2) 


